-- Logs begin at Tue 2022-07-05 09:25:06 UTC, end at Tue 2022-07-05 10:38:49 UTC. -- Jul 05 09:58:15.975726 dv-cinder-master-3-smb systemd[1]: Starting Devstack devstack@keystone.service... Jul 05 09:58:15.984993 dv-cinder-master-3-smb devstack@keystone.service[80363]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 5 09:58:15 2022] *** Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: nodename: dv-cinder-master-3-smb Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: machine: x86_64 Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: clock source: unix Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: pcre jit disabled Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: detected number of CPU cores: 4 Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: current working directory: / Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: detected binary path: /usr/bin/uwsgi-core Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: your processes number limit is 47680 Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: your memory page size is 4096 bytes Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: detected max file descriptor number: 2048 Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: lock engine: pthread robust mutexes Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: thunder lock: enabled Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 05 09:58:15.988118 dv-cinder-master-3-smb devstack@keystone.service[80363]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 05 09:58:16.030045 dv-cinder-master-3-smb devstack@keystone.service[80363]: Python main interpreter initialized at 0x561cf37c29c0 Jul 05 09:58:16.030045 dv-cinder-master-3-smb devstack@keystone.service[80363]: python threads support enabled Jul 05 09:58:16.030045 dv-cinder-master-3-smb devstack@keystone.service[80363]: your server socket listen backlog is limited to 100 connections Jul 05 09:58:16.030045 dv-cinder-master-3-smb devstack@keystone.service[80363]: your mercy for graceful operations on workers is 90 seconds Jul 05 09:58:16.030405 dv-cinder-master-3-smb devstack@keystone.service[80363]: mapped 403077 bytes (393 KB) for 2 cores Jul 05 09:58:16.030405 dv-cinder-master-3-smb devstack@keystone.service[80363]: *** Operational MODE: preforking *** Jul 05 09:58:16.030699 dv-cinder-master-3-smb devstack@keystone.service[80363]: *** uWSGI is running in multiple interpreter mode *** Jul 05 09:58:16.030699 dv-cinder-master-3-smb devstack@keystone.service[80363]: spawned uWSGI master process (pid: 80363) Jul 05 09:58:16.030917 dv-cinder-master-3-smb systemd[1]: Started Devstack devstack@keystone.service. Jul 05 09:58:16.031592 dv-cinder-master-3-smb devstack@keystone.service[80363]: spawned uWSGI worker 1 (pid: 80376, cores: 1) Jul 05 09:58:16.032706 dv-cinder-master-3-smb devstack@keystone.service[80363]: spawned uWSGI worker 2 (pid: 80377, cores: 1) Jul 05 09:58:16.032706 dv-cinder-master-3-smb devstack@keystone.service[80363]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 05 09:58:21.370553 dv-cinder-master-3-smb devstack@keystone.service[80377]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 09:58:21.370553 dv-cinder-master-3-smb devstack@keystone.service[80377]: from cryptography.utils import int_from_bytes Jul 05 09:58:21.371388 dv-cinder-master-3-smb devstack@keystone.service[80377]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 09:58:21.371388 dv-cinder-master-3-smb devstack@keystone.service[80377]: from cryptography.utils import int_from_bytes Jul 05 09:58:21.371504 dv-cinder-master-3-smb devstack@keystone.service[80376]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 09:58:21.371504 dv-cinder-master-3-smb devstack@keystone.service[80376]: from cryptography.utils import int_from_bytes Jul 05 09:58:21.371504 dv-cinder-master-3-smb devstack@keystone.service[80376]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 09:58:21.371504 dv-cinder-master-3-smb devstack@keystone.service[80376]: from cryptography.utils import int_from_bytes Jul 05 09:58:21.420158 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 05 09:58:21.420340 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 05 09:58:21.420472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 05 09:58:21.420564 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] command line args: [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 05 09:58:21.420689 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 05 09:58:21.420810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 05 09:58:21.420901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] command line args: [] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 05 09:58:21.421022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ================================================================================ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 05 09:58:21.421183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 05 09:58:21.421319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] admin_token = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.421398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ================================================================================ {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 05 09:58:21.421583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] config_dir = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.421715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] admin_token = **** {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.421791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.421852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] config_dir = [] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.421982 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] config_source = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.422065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.422140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] config_source = [] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.422204 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] debug = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.422363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] debug = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.422452 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.422707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.422806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.422877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.423012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.423092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.423294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.423427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.423497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.423557 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.423758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] list_limit = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.423908 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.424022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] log_config_append = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.424137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] log_config_append = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.424238 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.424341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.424440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] log_dir = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.424440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] log_file = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.424633 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] log_dir = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.424633 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] log_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.424820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.424880 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.424977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.425242 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.425375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.425462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.425539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.425602 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.425815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.426005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.426005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.426290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.426438 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.426549 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.426654 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.426763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.426911 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.427018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.427146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.427252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.427358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.427462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.427602 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.427713 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.427824 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.427956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.428061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.428164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.428307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.428307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.428502 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.428502 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] publish_errors = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.428748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.428883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.428993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] publish_errors = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.429128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.429240 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.429351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.429455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.429555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.429629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.429725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.429837 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.429915 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] standard_threads = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.429991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.430062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.430265 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] standard_threads = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.430367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.430445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.430516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.430585 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] use_journal = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.430653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.430774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] use_json = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.430899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.430980 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] use_journal = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.431050 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] use_stderr = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.431121 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] use_json = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.431198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] use_syslog = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.431280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] use_stderr = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.431357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.431441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] use_syslog = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.431587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.431710 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 09:58:21.431788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.431873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.432073 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.432240 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.432333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.432415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.432528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.432623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.432796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.432917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.433036 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.433123 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.433301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.433435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.433514 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.433585 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.433769 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.433891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.433979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.434063 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.434252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.434353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.434487 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.434568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.434729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.434821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.434981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.435057 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.435202 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.435286 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.435443 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.435615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.435696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.435780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.435924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.436013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.436180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.436270 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.436417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.436500 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.436648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.436727 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.436901 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.437018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.437262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.437342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.437518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.437598 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.437755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.437835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.437990 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.438077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.438234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.438342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.438521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.438614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.438739 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.438820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.438980 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.439058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.439197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.439281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.439431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.439509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.439663 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.439741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.439918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.439995 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.440155 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.440230 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.440392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.440466 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.440630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.440712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.440887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.440961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.441122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.441198 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.441344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.441416 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.441571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.441648 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.441800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.441876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.442029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.442112 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.442280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.442360 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.442519 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.442595 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.442757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.442869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.442988 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.443062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.443219 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.443293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.443448 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.443525 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.443684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.443760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.443920 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.443995 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.444154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.444230 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.444388 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.444463 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.444626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.444731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.444860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.444934 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.445095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.445166 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.445333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.445406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.445681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.445800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.445800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.446002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.446106 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.446264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.446395 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.446518 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.446625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.446770 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.446885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.447024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.447136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.447245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.447387 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.447498 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.447604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.447741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.447848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.447961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.448094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.448198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.448303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.448407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.448538 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.448650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.448795 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.448914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.449020 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.449151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.449254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.449366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.449498 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.449601 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.449710 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.449841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.449947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.450059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.450164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.450296 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.450402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.450475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.450545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.450661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.450736 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.450806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.450876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.450983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.451060 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.451139 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.451312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.451438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.451527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.451631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.451752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.451910 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.452033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.452121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.452222 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.452375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.452499 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.452662 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.452740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.452811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.452888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.453041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.453158 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.453285 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.453380 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.453522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.453688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.453819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.453941 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.454054 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.454177 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.454309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.454422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.454551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.454670 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.454670 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.454888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.455001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.455111 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.455235 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.455689 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.455817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.455939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.456053 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.456163 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.456275 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.456391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.456584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.456705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.456817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.external = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.456946 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] auth.external = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.457063 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.457175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.457291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.457436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.457566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.457687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.457800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.password = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.457915 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] auth.password = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.458025 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] auth.token = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.458143 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] auth.token = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.458309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.458438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.458552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.458662 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.458773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.458883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.458994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.459112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.459227 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.459338 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.459456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.459568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.459679 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.459797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.459909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.460020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] credential.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.460130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.460246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.460359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.460483 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.460597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.460716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.460834 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.461004 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.461124 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.461245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.461357 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.461468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.461580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.461699 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.461811 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.461930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.462043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.462154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.462278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.462391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.462502 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.462660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.462779 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.462890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.463002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.463113 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.463223 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.463345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.463457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.463568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.463680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.463791 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] federation.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.463902 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.464022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.464134 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.464245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.464357 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.464468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.464580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.464700 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.464813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.464926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.465038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.465149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.465260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.465380 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.465492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.465606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.465718 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.465829 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.465948 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.466070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.466182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.466320 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.466433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.466824 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.467024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.467139 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.467257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.467370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.467482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.467593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.467712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.467826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.467943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.468054 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.468165 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.468282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.468403 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.468516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.468633 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.468745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.468866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.468979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.469098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.469212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.469323 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.469434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.469548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.469658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.469782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.469896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.470008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.470124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.470251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.470365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.470488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.470602 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.470728 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.470861 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.470973 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.470973 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.471198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.471316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.471428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.471539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.471647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.471758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.471877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.471990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.472100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.472211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.472321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.472433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.472551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.472663 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.472773 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.472884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.472994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.473112 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.473232 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.473344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.473454 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.473565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.473675 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.473785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.473906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.474019 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.474131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.474326 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.474441 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.474552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.474675 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.474788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.474898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.475008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.475118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.475229 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.475347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.475460 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.475570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.475680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.475790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.475900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.476019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.476137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.476250 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.476363 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.476473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.476583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.476701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.476813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.476924 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.477034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.477144 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.477263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.477381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.477492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.477603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.477713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.477823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.477981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.478095 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.478206 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.478339 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.478450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.478561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.478684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.478797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.478910 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.479021 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.479138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.479249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.479370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.479482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.479599 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.479711 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.479822 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.479931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.480050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.480162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.480273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.480415 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.480531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.480642 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.480763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.480875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.480986 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.481096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.481206 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.481316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.481451 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.481609 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.481723 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.481834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.481944 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.482055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.482179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.482347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.482460 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.482571 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.482682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.482792 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.482917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.483031 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.483141 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.483141 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.483348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.483459 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.483578 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.483691 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.483802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.483915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.484025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.484025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.484239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.484239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.484447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.484558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.484668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.484778 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.484897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.485011 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.485122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.485233 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.485343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.485453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.485572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.485686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.485796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.485907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.486018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.486128 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.486267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.486381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.486492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.486602 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.486713 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.486824 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.486943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.487058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.487169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.487280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.487390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] resource.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.487501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.487650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.487767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.487882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.487994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.488104 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.488215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.488333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.488461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.488614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.488721 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.488833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.488943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.489129 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.489243 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] role.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.489354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.489465 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] role.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.489575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.489685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.489803 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] role.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.489916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.490070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] role.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.490070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.490303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.490303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.490527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.490637 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.490747 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.490860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.490972 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.491090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.491202 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.491313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.491423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.491534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.491643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.491761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.491873 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.491984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.492094 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.492204 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.492315 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.492433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.492545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.492692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.492834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.492834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.493043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.493162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.493274 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.493274 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.493481 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.493591 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.493701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.493819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.493931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.494073 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.494185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.494333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.494447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.494566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.494678 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.494790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.494901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.495011 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.495133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.495245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.495356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.495466 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.495577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.495688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.495806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.495918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.496029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.496140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.496250 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.496361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.496478 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.496590 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.496701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.496812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.496922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.497033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.497151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.497264 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] token.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.497374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.497484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.497596 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.497707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.497825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] token.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.497942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.498052 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.498163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.498306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.498418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.498542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.498654 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.498764 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.498876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.498985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.499096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.499214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.499327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.499437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.499547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.499687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.499799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.499918 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.500030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.500140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.500251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.500361 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.500472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.500590 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.500702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.500813 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.500952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.501065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.501175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.501294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.501407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.501530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.501651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.501763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.501882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.501993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.connection = **** {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.502116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.502282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.502409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.502523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.502634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.502745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.502856 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.502967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.503085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.503219 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.503334 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.503446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.503557 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.503668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.503788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.503901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.504012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.504123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.504234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.504345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.504464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.504579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.504690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.504801 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.504912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.505023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.505142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.505255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.505367 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.505478 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.505590 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.505590 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 05 09:58:21.505811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.505811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.505811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 09:58:21.506118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 05 09:58:21.536506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.536736 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.536976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.542462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.542751 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.543034 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.545306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.551828 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.573278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.573509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.579268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.579512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.721008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.731088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.769709 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.769943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.775769 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.776088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.833618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.833844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.834067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.834211 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.834346 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.834412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.834481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.834630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.834756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.834853 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.835060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.835242 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.858793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.859024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.859166 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.859257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.859334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.859441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.859535 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.859682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.859806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.859884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.859961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.860075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.860164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.860386 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.871166 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.871674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.884068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.884299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.884498 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.884599 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.884724 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.884825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.884967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.885109 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.885192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.885269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.885340 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.885413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.885574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.885744 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 09:58:21.902793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.902969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 09:58:21.903161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.903330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.903457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 09:58:21.903543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 09:58:21.903708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.903839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.903917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 09:58:21.904043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.904134 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.904330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.904462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.904546 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.904627 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.904892 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.915424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.915708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.915833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.915957 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.916071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.916245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.916370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.916526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.924482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.924729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.927832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.928041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.928168 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.928288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.928379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.928593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.928719 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.928874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.928999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.929149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.929290 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.929370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.929446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.929657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.929796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.929880 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.929971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.930179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.930333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.930455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.930546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.930746 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.957425 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.958502 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.959060 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.959338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.960088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.960367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.964596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.964887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.965281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.965592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.965874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.966149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.972751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.972972 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.973554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.973837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.977437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.977729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.978009 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.978284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:21.978474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.978567 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.978639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.978860 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.978987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.979133 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.979243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.979392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.979519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.979632 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.979918 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.980177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.997949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.998242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.998754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:21.999035 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.002772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.003558 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.006552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.006848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.007244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.007534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.011968 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.012624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.015575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.016281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.019247 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.019551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.019830 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.019954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.020129 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.020268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.020391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.020484 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.020672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.020766 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.021028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.021312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.039742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.040022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.040253 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.040384 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.040482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.040563 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.040779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.041040 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.048656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.049076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.050195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.050507 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.050611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.050895 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.053727 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.054007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.054124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.054309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.054399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.054549 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.054647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.054866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.054959 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.055106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.068857 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.069113 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.069425 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.069630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.069723 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.069891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.069976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.070201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.070544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.070782 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.071145 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.071428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.071819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.072118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.090547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.093446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.093880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.096734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.097326 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.099106 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.100112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.100743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.101023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.101318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.101701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.103497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.103776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.104079 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.108609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.111368 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.111499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.113035 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.114681 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.114966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.115355 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.115646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.116327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.116619 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.116900 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.117172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.124751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.125038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.125320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.126364 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.126662 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.126937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.133201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.133485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.133763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.134046 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.134356 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.134734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.134859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 09:58:22.134993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.135085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.135307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.135433 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.135556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.135656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.135853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.135978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.136135 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.136274 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.136409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.136487 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.136649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.136775 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.136896 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.136999 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.137266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.137559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.137806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.138070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.138374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 09:58:22.174713 dv-cinder-master-3-smb devstack@keystone.service[80377]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561cf37c29c0 pid: 80377 (default app) Jul 05 09:58:22.175788 dv-cinder-master-3-smb devstack@keystone.service[80376]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561cf37c29c0 pid: 80376 (default app) Jul 05 09:58:22.178244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c2128-22bd-48fd-8ed5-7fddd594f24f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:22.178558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c2128-22bd-48fd-8ed5-7fddd594f24f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:22.178847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c2128-22bd-48fd-8ed5-7fddd594f24f None None] PATH_INFO: `/v3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:22.183105 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Tue Jul 5 09:58:22 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:29.844455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bab65-ac78-4616-a868-aa1cd9d77a5a None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:29.845406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bab65-ac78-4616-a868-aa1cd9d77a5a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:29.845406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bab65-ac78-4616-a868-aa1cd9d77a5a None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:29.852521 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:29 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:29.861502 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9f805-d25e-421d-a40f-b947e852b29e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:29.861936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9f805-d25e-421d-a40f-b947e852b29e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:29.862077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9f805-d25e-421d-a40f-b947e852b29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:29.932208 dv-cinder-master-3-smb devstack@keystone.service[80377]: INFO dbcounter [None req-1fb9f805-d25e-421d-a40f-b947e852b29e None None] Registered counter for database keystone Jul 05 09:58:29.957670 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG oslo_db.sqlalchemy.engines [None req-1fb9f805-d25e-421d-a40f-b947e852b29e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80377) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 09:58:29.965291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writer thread running {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 09:58:30.144614 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG passlib.handlers.bcrypt [None req-1fb9f805-d25e-421d-a40f-b947e852b29e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80377) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 05 09:58:30.144961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG passlib.handlers.bcrypt [None req-1fb9f805-d25e-421d-a40f-b947e852b29e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80377) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 05 09:58:30.200043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1fb9f805-d25e-421d-a40f-b947e852b29e None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:30.201690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1fb9f805-d25e-421d-a40f-b947e852b29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:30.345210 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 05 09:58:30.573211 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bab65-ac78-4616-a868-aa1cd9d77a5a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:30.647469 dv-cinder-master-3-smb devstack@keystone.service[80376]: INFO dbcounter [None req-274bab65-ac78-4616-a868-aa1cd9d77a5a None None] Registered counter for database keystone Jul 05 09:58:30.668650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG oslo_db.sqlalchemy.engines [None req-274bab65-ac78-4616-a868-aa1cd9d77a5a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80376) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 09:58:30.679644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writer thread running {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 09:58:30.796569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bab65-ac78-4616-a868-aa1cd9d77a5a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:30.801719 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c5ba7-cc2b-4403-aaa4-d86925ea5881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:30.802475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c5ba7-cc2b-4403-aaa4-d86925ea5881 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:30.802757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c5ba7-cc2b-4403-aaa4-d86925ea5881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:30.803041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c5ba7-cc2b-4403-aaa4-d86925ea5881 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:30.811707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c5ba7-cc2b-4403-aaa4-d86925ea5881 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:30.864979 dv-cinder-master-3-smb devstack@keystone.service[80376]: WARNING keystone.common.rbac_enforcer.enforcer [None req-245c5ba7-cc2b-4403-aaa4-d86925ea5881 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 05 09:58:30.865565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c5ba7-cc2b-4403-aaa4-d86925ea5881 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:30.875770 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Tue Jul 5 09:58:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 09:58:30.882400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9f805-d25e-421d-a40f-b947e852b29e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:30.898552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9f805-d25e-421d-a40f-b947e852b29e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:30.904219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3d5cb-4ddf-44ac-ade4-e6adac2e4c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:30.905377 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d5cb-4ddf-44ac-ade4-e6adac2e4c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:30.905667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d5cb-4ddf-44ac-ade4-e6adac2e4c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:30.905939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d5cb-4ddf-44ac-ade4-e6adac2e4c59 admin admin] PATH_INFO: `/v3/projects/95874cd0621e46bbbfb452bfbd474389` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:30.907625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3d5cb-4ddf-44ac-ade4-e6adac2e4c59 admin admin] RBAC: Authorizing `identity:get_project(project_id=95874cd0621e46bbbfb452bfbd474389)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:30.961464 dv-cinder-master-3-smb devstack@keystone.service[80377]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1bf3d5cb-4ddf-44ac-ade4-e6adac2e4c59 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 05 09:58:30.961806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3d5cb-4ddf-44ac-ade4-e6adac2e4c59 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:30.966402 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Tue Jul 5 09:58:30 2022] GET /identity/v3/projects/95874cd0621e46bbbfb452bfbd474389 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:32.948940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b4ffc-cb6a-4adf-a668-b89d8f0b7aee None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:32.949516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b4ffc-cb6a-4adf-a668-b89d8f0b7aee None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:32.949516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b4ffc-cb6a-4adf-a668-b89d8f0b7aee None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:32.953841 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:32.961122 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39667c68-ddf2-4281-9954-a79547fd387e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:32.961832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39667c68-ddf2-4281-9954-a79547fd387e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:32.961996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39667c68-ddf2-4281-9954-a79547fd387e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:32.987629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-39667c68-ddf2-4281-9954-a79547fd387e None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:32.989165 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-39667c68-ddf2-4281-9954-a79547fd387e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:32.998685 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 05 09:58:33.021306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b4ffc-cb6a-4adf-a668-b89d8f0b7aee None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:33.035722 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b4ffc-cb6a-4adf-a668-b89d8f0b7aee None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:33.041283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe267de-c518-4fa2-9b31-4d7617337e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:33.041913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe267de-c518-4fa2-9b31-4d7617337e96 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:33.042174 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe267de-c518-4fa2-9b31-4d7617337e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:33.042491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe267de-c518-4fa2-9b31-4d7617337e96 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:33.051069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe267de-c518-4fa2-9b31-4d7617337e96 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:33.052894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe267de-c518-4fa2-9b31-4d7617337e96 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:33.062279 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Tue Jul 5 09:58:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 09:58:33.070102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39667c68-ddf2-4281-9954-a79547fd387e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:33.079674 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39667c68-ddf2-4281-9954-a79547fd387e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:33.086454 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dbeaf-383b-40ba-8837-73750b05ba7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:33.087344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dbeaf-383b-40ba-8837-73750b05ba7c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:33.087622 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dbeaf-383b-40ba-8837-73750b05ba7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:33.087900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dbeaf-383b-40ba-8837-73750b05ba7c admin admin] PATH_INFO: `/v3/users/c042e1f397444e8991d3f0b7bfb28908` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:33.089457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dbeaf-383b-40ba-8837-73750b05ba7c admin admin] RBAC: Authorizing `identity:get_user(user_id=c042e1f397444e8991d3f0b7bfb28908)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:33.091338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dbeaf-383b-40ba-8837-73750b05ba7c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:33.092907 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Tue Jul 5 09:58:33 2022] GET /identity/v3/users/c042e1f397444e8991d3f0b7bfb28908 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:35.128604 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a8606-6c64-4710-ad51-5d85d5051ab0 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:35.129131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a8606-6c64-4710-ad51-5d85d5051ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:35.129131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a8606-6c64-4710-ad51-5d85d5051ab0 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:35.130486 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:35.135886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1794086-4dea-460c-9a02-5ae5f05b6da8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:35.136319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1794086-4dea-460c-9a02-5ae5f05b6da8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:35.136523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1794086-4dea-460c-9a02-5ae5f05b6da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:35.154343 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f1794086-4dea-460c-9a02-5ae5f05b6da8 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:35.155534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f1794086-4dea-460c-9a02-5ae5f05b6da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:35.174023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0675c1-9eba-4202-acec-00fb9859ad71 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:35.174413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0675c1-9eba-4202-acec-00fb9859ad71 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:35.174776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0675c1-9eba-4202-acec-00fb9859ad71 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:35.176165 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 09:58:35.181245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b1fbf-7ffa-4444-9658-5f6e7dc54970 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:35.181524 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b1fbf-7ffa-4444-9658-5f6e7dc54970 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:35.181797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b1fbf-7ffa-4444-9658-5f6e7dc54970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:35.221960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG passlib.handlers.bcrypt [None req-013b1fbf-7ffa-4444-9658-5f6e7dc54970 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80376) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 05 09:58:35.222431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG passlib.handlers.bcrypt [None req-013b1fbf-7ffa-4444-9658-5f6e7dc54970 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80376) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 05 09:58:35.239477 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:35.262375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1794086-4dea-460c-9a02-5ae5f05b6da8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:35.267521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-013b1fbf-7ffa-4444-9658-5f6e7dc54970 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:35.269100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-013b1fbf-7ffa-4444-9658-5f6e7dc54970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:35.287688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:35.365940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1794086-4dea-460c-9a02-5ae5f05b6da8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:35.368652 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:35.389985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b1fbf-7ffa-4444-9658-5f6e7dc54970 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:35.406921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77c14a-faf8-4b60-852e-2d4bb2bc769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:35.407743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77c14a-faf8-4b60-852e-2d4bb2bc769f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:35.408027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77c14a-faf8-4b60-852e-2d4bb2bc769f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:35.408303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77c14a-faf8-4b60-852e-2d4bb2bc769f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:35.419555 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77c14a-faf8-4b60-852e-2d4bb2bc769f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-be77c14a-faf8-4b60-852e-2d4bb2bc769f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:35.430874 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:35.431926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:58:35.432954 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:58:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:35.437840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77c14a-faf8-4b60-852e-2d4bb2bc769f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:35.490171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b1fbf-7ffa-4444-9658-5f6e7dc54970 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:35.492449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:35.521531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:35.527571 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad925a13-1a5b-406d-b2ba-1eae27601546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:35.528529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad925a13-1a5b-406d-b2ba-1eae27601546 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:35.528831 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad925a13-1a5b-406d-b2ba-1eae27601546 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:35.529112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad925a13-1a5b-406d-b2ba-1eae27601546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:35.539852 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77c14a-faf8-4b60-852e-2d4bb2bc769f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:35.573718 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638993-482e-4597-86cf-66fe1edba9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:35.574441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638993-482e-4597-86cf-66fe1edba9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:35.574729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638993-482e-4597-86cf-66fe1edba9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:35.575005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638993-482e-4597-86cf-66fe1edba9c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:35.575771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26638993-482e-4597-86cf-66fe1edba9c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:35.576085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26638993-482e-4597-86cf-66fe1edba9c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:35.577508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26638993-482e-4597-86cf-66fe1edba9c8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:35.590086 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:35.595813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638993-482e-4597-86cf-66fe1edba9c8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:35.599970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad925a13-1a5b-406d-b2ba-1eae27601546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:35.601510 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad925a13-1a5b-406d-b2ba-1eae27601546 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:35.694035 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638993-482e-4597-86cf-66fe1edba9c8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:35.699968 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:58:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 09:58:35.705741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad925a13-1a5b-406d-b2ba-1eae27601546 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:35.709815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:35.730025 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f662663-ec09-4fac-98f4-0b92e927afcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:35.731562 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f662663-ec09-4fac-98f4-0b92e927afcb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:35.732077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f662663-ec09-4fac-98f4-0b92e927afcb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:35.732373 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f662663-ec09-4fac-98f4-0b92e927afcb None admin] PATH_INFO: `/v3/users/c042e1f397444e8991d3f0b7bfb28908` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:35.733678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f662663-ec09-4fac-98f4-0b92e927afcb None admin] RBAC: Authorizing `identity:get_user(user_id=c042e1f397444e8991d3f0b7bfb28908)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:35.735177 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f662663-ec09-4fac-98f4-0b92e927afcb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:35.736836 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:35 2022] GET /identity/v3/users/c042e1f397444e8991d3f0b7bfb28908 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:35.742865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f662663-ec09-4fac-98f4-0b92e927afcb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:35.759497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:35.807085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad925a13-1a5b-406d-b2ba-1eae27601546 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:35.841007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bc38a-83c0-47af-9fa3-17d5a92baaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:35.841823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bc38a-83c0-47af-9fa3-17d5a92baaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:35.842107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bc38a-83c0-47af-9fa3-17d5a92baaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:35.842412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bc38a-83c0-47af-9fa3-17d5a92baaa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:35.842653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f662663-ec09-4fac-98f4-0b92e927afcb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:35.861692 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bc38a-83c0-47af-9fa3-17d5a92baaa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:35.863206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bc38a-83c0-47af-9fa3-17d5a92baaa5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:35.875579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e259c8-1a57-4644-a012-734df7cdb54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:35.876387 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e259c8-1a57-4644-a012-734df7cdb54d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:35.876676 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e259c8-1a57-4644-a012-734df7cdb54d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:35.876955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e259c8-1a57-4644-a012-734df7cdb54d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:35.878152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e259c8-1a57-4644-a012-734df7cdb54d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:35.879621 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e259c8-1a57-4644-a012-734df7cdb54d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:35.881296 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 10/18] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:58:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-d90bc38a-83c0-47af-9fa3-17d5a92baaa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:58:35.883269 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:35.884270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:35.885412 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:58:35.885412 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:58:35.885412 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:58:35.885412 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:58:35.885412 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 9/19] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:58:35.888529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e259c8-1a57-4644-a012-734df7cdb54d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:35.889855 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bc38a-83c0-47af-9fa3-17d5a92baaa5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:35.943586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:35.967764 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:35.986725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e259c8-1a57-4644-a012-734df7cdb54d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:35.986863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bc38a-83c0-47af-9fa3-17d5a92baaa5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:36.018969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598080be-e25c-4223-8933-2c62e6fa6530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:36.019304 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d2355-2ab4-4902-9837-a048692d9872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:36.019622 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598080be-e25c-4223-8933-2c62e6fa6530 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:36.019905 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598080be-e25c-4223-8933-2c62e6fa6530 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:36.020006 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d2355-2ab4-4902-9837-a048692d9872 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:36.020182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598080be-e25c-4223-8933-2c62e6fa6530 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:36.020344 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d2355-2ab4-4902-9837-a048692d9872 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:36.020616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d2355-2ab4-4902-9837-a048692d9872 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:36.021039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598080be-e25c-4223-8933-2c62e6fa6530 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.domain.id=default, user.id=c042e1f397444e8991d3f0b7bfb28908) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:36.021348 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598080be-e25c-4223-8933-2c62e6fa6530 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:36.021448 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d2355-2ab4-4902-9837-a048692d9872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:36.021736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d2355-2ab4-4902-9837-a048692d9872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:36.022824 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598080be-e25c-4223-8933-2c62e6fa6530 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:36.023199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d2355-2ab4-4902-9837-a048692d9872 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:36.029910 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1322 bytes} [Tue Jul 5 09:58:35 2022] GET /identity/v3/role_assignments?user.id=c042e1f397444e8991d3f0b7bfb28908&scope.domain.id=default&role.id=c184a177010144429f495f3290d28221 => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:36.041112 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:38.330634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d494e7-0476-4753-8c83-c7b0f23618a3 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:38.331792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d494e7-0476-4753-8c83-c7b0f23618a3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:38.332236 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d494e7-0476-4753-8c83-c7b0f23618a3 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:38.333798 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:38.351877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ec923-02dd-4d04-99d6-b636dc43475b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:38.352356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ec923-02dd-4d04-99d6-b636dc43475b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:38.352428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ec923-02dd-4d04-99d6-b636dc43475b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:38.377458 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-bb4ec923-02dd-4d04-99d6-b636dc43475b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:38.378710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-bb4ec923-02dd-4d04-99d6-b636dc43475b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:38.459790 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 11/23] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:38.494328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d494e7-0476-4753-8c83-c7b0f23618a3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:38.540098 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97113b21-fae4-4d15-bbd9-771a3284d8b8 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:38.540387 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97113b21-fae4-4d15-bbd9-771a3284d8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:38.540906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97113b21-fae4-4d15-bbd9-771a3284d8b8 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:38.542500 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 12/24] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 09:58:38.551399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5d975-9e55-4545-82a0-947bc8dad853 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:38.551935 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5d975-9e55-4545-82a0-947bc8dad853 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:38.552447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5d975-9e55-4545-82a0-947bc8dad853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:38.574480 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-adf5d975-9e55-4545-82a0-947bc8dad853 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:38.578436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-adf5d975-9e55-4545-82a0-947bc8dad853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:38.689593 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 13/25] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:58:38.693369 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:38.701612 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac289ed-f96e-402b-9c24-7cbd70bd0472 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:38.702153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac289ed-f96e-402b-9c24-7cbd70bd0472 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:38.702677 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac289ed-f96e-402b-9c24-7cbd70bd0472 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:38.703270 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d494e7-0476-4753-8c83-c7b0f23618a3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:38.704330 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 14/26] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 09:58:38.720295 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cc25c-5b68-4bcc-9439-c956b18491f2 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:38.720898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cc25c-5b68-4bcc-9439-c956b18491f2 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:38.721447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cc25c-5b68-4bcc-9439-c956b18491f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:38.750269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-bd5cc25c-5b68-4bcc-9439-c956b18491f2 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:38.750899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73cbf8-484f-41f8-9f76-1bf4165881ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:38.751713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73cbf8-484f-41f8-9f76-1bf4165881ea None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:38.751994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73cbf8-484f-41f8-9f76-1bf4165881ea None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:38.752266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73cbf8-484f-41f8-9f76-1bf4165881ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:38.753934 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-bd5cc25c-5b68-4bcc-9439-c956b18491f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:38.773699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73cbf8-484f-41f8-9f76-1bf4165881ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ab73cbf8-484f-41f8-9f76-1bf4165881ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:38.781396 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:38.782267 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:38.783032 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:58:38.784044 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 13/27] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:58:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:38.790455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73cbf8-484f-41f8-9f76-1bf4165881ea None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:38.802605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:38.903364 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 15/28] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:58:38.904572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cc25c-5b68-4bcc-9439-c956b18491f2 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:39.013841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73cbf8-484f-41f8-9f76-1bf4165881ea None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:39.054172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cc25c-5b68-4bcc-9439-c956b18491f2 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:39.064195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809f2e8-49d2-4a3a-beb7-f1e74fb6c063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:39.065623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809f2e8-49d2-4a3a-beb7-f1e74fb6c063 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:39.066105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809f2e8-49d2-4a3a-beb7-f1e74fb6c063 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:39.066559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:39.067173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809f2e8-49d2-4a3a-beb7-f1e74fb6c063 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:39.068350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809f2e8-49d2-4a3a-beb7-f1e74fb6c063 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:39.070239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809f2e8-49d2-4a3a-beb7-f1e74fb6c063 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:39.127344 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dc9b0-bab4-4d12-9ab8-702330d7fa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:39.128406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dc9b0-bab4-4d12-9ab8-702330d7fa7b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:39.128906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dc9b0-bab4-4d12-9ab8-702330d7fa7b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:39.129368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dc9b0-bab4-4d12-9ab8-702330d7fa7b None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:39.130455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dc9b0-bab4-4d12-9ab8-702330d7fa7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:39.134696 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dc9b0-bab4-4d12-9ab8-702330d7fa7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:39.136330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dc9b0-bab4-4d12-9ab8-702330d7fa7b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:39.172338 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 16/29] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:39.173240 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dc9b0-bab4-4d12-9ab8-702330d7fa7b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:39.251204 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 14/30] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Jul 5 09:58:38 2022] POST /identity/v3/roles => generated 225 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:58:39.252604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809f2e8-49d2-4a3a-beb7-f1e74fb6c063 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:39.341943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dc9b0-bab4-4d12-9ab8-702330d7fa7b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:39.350651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:39.423891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a23c8-a351-4f77-9fa2-2a40ac98bf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:39.424694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a23c8-a351-4f77-9fa2-2a40ac98bf19 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:39.424974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a23c8-a351-4f77-9fa2-2a40ac98bf19 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:39.425243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a23c8-a351-4f77-9fa2-2a40ac98bf19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:39.429361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809f2e8-49d2-4a3a-beb7-f1e74fb6c063 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:39.440148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a23c8-a351-4f77-9fa2-2a40ac98bf19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:39.441293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a23c8-a351-4f77-9fa2-2a40ac98bf19 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-e00a23c8-a351-4f77-9fa2-2a40ac98bf19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:58:39.461757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:39.462754 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:39.463577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:58:39.463577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:58:39.463577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:58:39.463577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:58:39.465603 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 17/31] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:39.467129 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96796dc3-0936-4183-bd91-3feb2821b525 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:39.467644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96796dc3-0936-4183-bd91-3feb2821b525 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:39.468151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96796dc3-0936-4183-bd91-3feb2821b525 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:39.469630 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 18/32] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:39.475455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a1904-f169-4855-bbd7-0d8e68e465ed None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:39.475968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a1904-f169-4855-bbd7-0d8e68e465ed None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:39.476539 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a1904-f169-4855-bbd7-0d8e68e465ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:39.481432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e4340-690b-4bdd-83ed-bb60b91b8481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:39.482355 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e4340-690b-4bdd-83ed-bb60b91b8481 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:39.482642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e4340-690b-4bdd-83ed-bb60b91b8481 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:39.482913 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e4340-690b-4bdd-83ed-bb60b91b8481 None admin] PATH_INFO: `/v3/users/c042e1f397444e8991d3f0b7bfb28908` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:39.484431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e4340-690b-4bdd-83ed-bb60b91b8481 None admin] RBAC: Authorizing `identity:get_user(user_id=c042e1f397444e8991d3f0b7bfb28908)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:39.486163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e4340-690b-4bdd-83ed-bb60b91b8481 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:39.488094 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 15/33] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:39 2022] GET /identity/v3/users/c042e1f397444e8991d3f0b7bfb28908 => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:39.489268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e4340-690b-4bdd-83ed-bb60b91b8481 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:39.498417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-192a1904-f169-4855-bbd7-0d8e68e465ed None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:39.499757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-192a1904-f169-4855-bbd7-0d8e68e465ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:39.516716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:39.578020 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 19/34] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:39.579290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a1904-f169-4855-bbd7-0d8e68e465ed None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:39.601913 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e4340-690b-4bdd-83ed-bb60b91b8481 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:39.632599 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:39.647091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65062bf6-2f41-4107-9e4e-af7322d74ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:39.648010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65062bf6-2f41-4107-9e4e-af7322d74ebb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:39.648452 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65062bf6-2f41-4107-9e4e-af7322d74ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:39.648872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65062bf6-2f41-4107-9e4e-af7322d74ebb None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:39.649807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65062bf6-2f41-4107-9e4e-af7322d74ebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:39.650313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65062bf6-2f41-4107-9e4e-af7322d74ebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:39.651888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65062bf6-2f41-4107-9e4e-af7322d74ebb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:39.674619 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 16/35] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:39.675796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65062bf6-2f41-4107-9e4e-af7322d74ebb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:39.706889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a1904-f169-4855-bbd7-0d8e68e465ed None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:39.744036 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42eb6d-8afd-41e2-83cd-6e9c63aa6ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:39.744943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42eb6d-8afd-41e2-83cd-6e9c63aa6ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:39.745175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42eb6d-8afd-41e2-83cd-6e9c63aa6ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:39.745455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42eb6d-8afd-41e2-83cd-6e9c63aa6ac9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:39.746970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42eb6d-8afd-41e2-83cd-6e9c63aa6ac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:39.748747 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42eb6d-8afd-41e2-83cd-6e9c63aa6ac9 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:39.750436 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 20/36] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:58:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:39.751185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42eb6d-8afd-41e2-83cd-6e9c63aa6ac9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:39.778977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:39.791370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65062bf6-2f41-4107-9e4e-af7322d74ebb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:39.825915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893c5e8-9734-4636-af65-07602b98ea34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:39.826640 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893c5e8-9734-4636-af65-07602b98ea34 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:39.826932 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893c5e8-9734-4636-af65-07602b98ea34 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:39.827212 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893c5e8-9734-4636-af65-07602b98ea34 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:39.827988 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893c5e8-9734-4636-af65-07602b98ea34 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:39.829407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893c5e8-9734-4636-af65-07602b98ea34 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:39.843362 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 17/37] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Jul 5 09:58:39 2022] POST /identity/v3/roles => generated 219 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:58:39.854022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01350d-8e8b-43d1-87c6-ec8d87df1cf3 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:39.854346 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01350d-8e8b-43d1-87c6-ec8d87df1cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:39.854634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01350d-8e8b-43d1-87c6-ec8d87df1cf3 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:39.857218 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 18/38] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:39.857218 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01350d-8e8b-43d1-87c6-ec8d87df1cf3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:39.867635 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42eb6d-8afd-41e2-83cd-6e9c63aa6ac9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:39.906846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1c05a-d1ea-4e65-9ffd-01b0224f37a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:39.907598 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1c05a-d1ea-4e65-9ffd-01b0224f37a6 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:39.907892 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1c05a-d1ea-4e65-9ffd-01b0224f37a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:39.908172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1c05a-d1ea-4e65-9ffd-01b0224f37a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:39.909238 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1c05a-d1ea-4e65-9ffd-01b0224f37a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:39.910426 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1c05a-d1ea-4e65-9ffd-01b0224f37a6 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:39.924688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:40.054865 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 21/39] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 09:58:39 2022] POST /identity/v3/projects => generated 301 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:58:40.056316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae268b2a-8e37-4f63-b24a-b23b03b78e88 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:40.056588 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae268b2a-8e37-4f63-b24a-b23b03b78e88 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:40.056871 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae268b2a-8e37-4f63-b24a-b23b03b78e88 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:40.058465 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 22/40] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:40.059432 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de436fc3-a7ca-471d-bd51-25eafd2aaadd None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:40.059723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de436fc3-a7ca-471d-bd51-25eafd2aaadd None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:40.060003 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de436fc3-a7ca-471d-bd51-25eafd2aaadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:40.078575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-de436fc3-a7ca-471d-bd51-25eafd2aaadd None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:40.080190 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-de436fc3-a7ca-471d-bd51-25eafd2aaadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:40.099580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01350d-8e8b-43d1-87c6-ec8d87df1cf3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:40.140258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31898086-0a0e-4ef2-a745-60b691c32aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:40.141546 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31898086-0a0e-4ef2-a745-60b691c32aab None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:40.141970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31898086-0a0e-4ef2-a745-60b691c32aab None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:40.142500 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31898086-0a0e-4ef2-a745-60b691c32aab None admin] PATH_INFO: `/v3/domains/default/users/c042e1f397444e8991d3f0b7bfb28908/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:40.145082 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31898086-0a0e-4ef2-a745-60b691c32aab None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c042e1f397444e8991d3f0b7bfb28908, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:40.147000 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31898086-0a0e-4ef2-a745-60b691c32aab None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:40.148761 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:40.154549 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 23/41] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:40.156577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aa49b-7ddd-4241-b972-9b73efc466ce None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:40.157092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aa49b-7ddd-4241-b972-9b73efc466ce None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:40.157370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aa49b-7ddd-4241-b972-9b73efc466ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:40.174815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-7a0aa49b-7ddd-4241-b972-9b73efc466ce None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:40.176261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7a0aa49b-7ddd-4241-b972-9b73efc466ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:40.183315 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 19/42] 10.222.0.16 () {58 vars in 1361 bytes} [Tue Jul 5 09:58:39 2022] PUT /identity/v3/domains/default/users/c042e1f397444e8991d3f0b7bfb28908/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:58:40.184518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31898086-0a0e-4ef2-a745-60b691c32aab None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:40.284588 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 24/43] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:40.309214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0aa49b-7ddd-4241-b972-9b73efc466ce None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:40.326563 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31898086-0a0e-4ef2-a745-60b691c32aab None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:40.342995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:40.376666 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1f65b-d6c3-4579-b3bd-4cf621091495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:40.377683 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1f65b-d6c3-4579-b3bd-4cf621091495 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:40.377998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1f65b-d6c3-4579-b3bd-4cf621091495 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:40.378598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1f65b-d6c3-4579-b3bd-4cf621091495 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:40.380126 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1f65b-d6c3-4579-b3bd-4cf621091495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:40.382150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1f65b-d6c3-4579-b3bd-4cf621091495 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:40.384196 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 20/44] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:58:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:40.390532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1f65b-d6c3-4579-b3bd-4cf621091495 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:40.424424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0aa49b-7ddd-4241-b972-9b73efc466ce None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:40.454996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:40.463100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253eba4-180f-4419-a818-cb3a59737808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:40.463776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253eba4-180f-4419-a818-cb3a59737808 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:40.464061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253eba4-180f-4419-a818-cb3a59737808 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:40.464337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253eba4-180f-4419-a818-cb3a59737808 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:40.465085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253eba4-180f-4419-a818-cb3a59737808 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:40.466645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253eba4-180f-4419-a818-cb3a59737808 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:40.482780 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 25/45] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Jul 5 09:58:40 2022] POST /identity/v3/roles => generated 223 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:58:40.514864 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1f65b-d6c3-4579-b3bd-4cf621091495 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:40.554013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877c851-2a02-45ad-861b-4783aaa73feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:40.554746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877c851-2a02-45ad-861b-4783aaa73feb None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:40.555036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877c851-2a02-45ad-861b-4783aaa73feb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:40.555320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877c851-2a02-45ad-861b-4783aaa73feb None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:40.556061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877c851-2a02-45ad-861b-4783aaa73feb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:40.557587 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877c851-2a02-45ad-861b-4783aaa73feb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:40.588566 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 21/46] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 09:58:40 2022] POST /identity/v3/projects => generated 312 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 09:58:42.241937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36b2b2-c195-4984-8e41-5b0bdb358cb7 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:42.242523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36b2b2-c195-4984-8e41-5b0bdb358cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:42.242523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36b2b2-c195-4984-8e41-5b0bdb358cb7 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:42.244007 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 26/47] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:42.249551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b82b5e-3642-4a5d-838f-120fc9f97c5c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:42.249931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b82b5e-3642-4a5d-838f-120fc9f97c5c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:42.250110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b82b5e-3642-4a5d-838f-120fc9f97c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:42.267839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-87b82b5e-3642-4a5d-838f-120fc9f97c5c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:42.269366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-87b82b5e-3642-4a5d-838f-120fc9f97c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:42.295299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:42.337473 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 22/48] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:42.360228 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36b2b2-c195-4984-8e41-5b0bdb358cb7 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:42.461790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36b2b2-c195-4984-8e41-5b0bdb358cb7 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:42.496924 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f7aec-6540-4887-9c13-762d85c5cbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:42.497789 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f7aec-6540-4887-9c13-762d85c5cbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:42.498140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f7aec-6540-4887-9c13-762d85c5cbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:42.498512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f7aec-6540-4887-9c13-762d85c5cbe0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:42.510896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f7aec-6540-4887-9c13-762d85c5cbe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-024f7aec-6540-4887-9c13-762d85c5cbe0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:42.519644 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:42.520564 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:58:42.521567 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 27/49] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:58:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:42.526952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b82b5e-3642-4a5d-838f-120fc9f97c5c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:42.595680 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51552963-d7e4-4735-88a8-89c2a587b797 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:42.595938 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51552963-d7e4-4735-88a8-89c2a587b797 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:42.596220 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51552963-d7e4-4735-88a8-89c2a587b797 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:42.597679 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 28/50] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:42.602695 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40ee19-aa6d-42f2-a4e1-2629e2e5c9ae None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:42.602988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40ee19-aa6d-42f2-a4e1-2629e2e5c9ae None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:42.603270 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40ee19-aa6d-42f2-a4e1-2629e2e5c9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:42.621802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-4b40ee19-aa6d-42f2-a4e1-2629e2e5c9ae None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:42.623076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4b40ee19-aa6d-42f2-a4e1-2629e2e5c9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:42.629266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b82b5e-3642-4a5d-838f-120fc9f97c5c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:42.665409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a9a34-f0ec-4f4b-a2bc-8d6d253cdfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:42.666101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a9a34-f0ec-4f4b-a2bc-8d6d253cdfec None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:42.666422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a9a34-f0ec-4f4b-a2bc-8d6d253cdfec None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:42.666704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a9a34-f0ec-4f4b-a2bc-8d6d253cdfec None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:42.667477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a9a34-f0ec-4f4b-a2bc-8d6d253cdfec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:42.667790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a9a34-f0ec-4f4b-a2bc-8d6d253cdfec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:42.669233 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a9a34-f0ec-4f4b-a2bc-8d6d253cdfec None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:42.679087 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 23/51] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:42.685146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a9a34-f0ec-4f4b-a2bc-8d6d253cdfec None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:42.714537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:42.777235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:42.800068 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 29/52] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:42.801473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a9a34-f0ec-4f4b-a2bc-8d6d253cdfec None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:42.823590 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40ee19-aa6d-42f2-a4e1-2629e2e5c9ae None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:42.834277 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbbaaf-72a5-4613-90ff-c19879f1254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:42.835171 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbbaaf-72a5-4613-90ff-c19879f1254a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:42.835458 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbbaaf-72a5-4613-90ff-c19879f1254a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:42.835770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbbaaf-72a5-4613-90ff-c19879f1254a None admin] PATH_INFO: `/v3/users/c042e1f397444e8991d3f0b7bfb28908` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:42.836991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbbaaf-72a5-4613-90ff-c19879f1254a None admin] RBAC: Authorizing `identity:get_user(user_id=c042e1f397444e8991d3f0b7bfb28908)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:42.838512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbbaaf-72a5-4613-90ff-c19879f1254a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:42.840189 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 24/53] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:42 2022] GET /identity/v3/users/c042e1f397444e8991d3f0b7bfb28908 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:42.845660 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbbaaf-72a5-4613-90ff-c19879f1254a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:42.919091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40ee19-aa6d-42f2-a4e1-2629e2e5c9ae None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:42.935005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:42.944954 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbbaaf-72a5-4613-90ff-c19879f1254a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:42.953515 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081fbd9-3045-4213-87f4-5932575170a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:42.954364 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081fbd9-3045-4213-87f4-5932575170a6 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:42.954894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081fbd9-3045-4213-87f4-5932575170a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:42.955178 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081fbd9-3045-4213-87f4-5932575170a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:42.956359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081fbd9-3045-4213-87f4-5932575170a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:42.958177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081fbd9-3045-4213-87f4-5932575170a6 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:42.959834 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 30/54] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:58:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:42.965260 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081fbd9-3045-4213-87f4-5932575170a6 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:42.977551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09309c31-fb1a-4373-866a-9eab71121c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:42.978402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09309c31-fb1a-4373-866a-9eab71121c7c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:42.978690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09309c31-fb1a-4373-866a-9eab71121c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:42.978967 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09309c31-fb1a-4373-866a-9eab71121c7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:42.980136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09309c31-fb1a-4373-866a-9eab71121c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:42.981575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09309c31-fb1a-4373-866a-9eab71121c7c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:42.983215 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 25/55] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:58:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:42.988769 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09309c31-fb1a-4373-866a-9eab71121c7c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:43.015164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:43.063660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081fbd9-3045-4213-87f4-5932575170a6 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:43.087825 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09309c31-fb1a-4373-866a-9eab71121c7c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:43.108429 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635af41f-5b6d-4e30-8a27-282be058a4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:43.109112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635af41f-5b6d-4e30-8a27-282be058a4ca None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:43.109402 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635af41f-5b6d-4e30-8a27-282be058a4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:43.109685 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635af41f-5b6d-4e30-8a27-282be058a4ca None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:43.110441 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635af41f-5b6d-4e30-8a27-282be058a4ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:43.111917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635af41f-5b6d-4e30-8a27-282be058a4ca None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:43.123910 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff19b0d-069e-49c6-82d4-30e76a546c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:43.124584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff19b0d-069e-49c6-82d4-30e76a546c1d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:43.124855 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff19b0d-069e-49c6-82d4-30e76a546c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:43.125136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff19b0d-069e-49c6-82d4-30e76a546c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:43.125997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff19b0d-069e-49c6-82d4-30e76a546c1d None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.domain.id=default, user.id=c042e1f397444e8991d3f0b7bfb28908) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:43.126332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff19b0d-069e-49c6-82d4-30e76a546c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:43.127924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff19b0d-069e-49c6-82d4-30e76a546c1d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:43.134787 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 26/56] 10.222.0.16 () {56 vars in 1322 bytes} [Tue Jul 5 09:58:42 2022] GET /identity/v3/role_assignments?user.id=c042e1f397444e8991d3f0b7bfb28908&scope.domain.id=default&role.id=c184a177010144429f495f3290d28221 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:43.138259 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 31/57] 10.222.0.16 () {60 vars in 1140 bytes} [Tue Jul 5 09:58:42 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:58:45.194985 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16653d69-08ca-430e-8ac3-3146d9fa52df None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:45.195874 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16653d69-08ca-430e-8ac3-3146d9fa52df None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:45.195874 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16653d69-08ca-430e-8ac3-3146d9fa52df None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:45.197922 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 27/58] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:45.206617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed22ab-111c-4bc9-b64a-81d6feca1140 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:45.207343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed22ab-111c-4bc9-b64a-81d6feca1140 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:45.207555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed22ab-111c-4bc9-b64a-81d6feca1140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:45.231557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-58ed22ab-111c-4bc9-b64a-81d6feca1140 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:45.232745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-58ed22ab-111c-4bc9-b64a-81d6feca1140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:45.302370 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 32/59] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:45.326726 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16653d69-08ca-430e-8ac3-3146d9fa52df None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:45.363162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:45.442372 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16653d69-08ca-430e-8ac3-3146d9fa52df None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:45.481715 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4abb79-f90b-4eda-9c51-d2ffd4af9822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:45.482470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4abb79-f90b-4eda-9c51-d2ffd4af9822 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:45.482763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4abb79-f90b-4eda-9c51-d2ffd4af9822 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:45.483046 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4abb79-f90b-4eda-9c51-d2ffd4af9822 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:45.484230 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4abb79-f90b-4eda-9c51-d2ffd4af9822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:45.485702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4abb79-f90b-4eda-9c51-d2ffd4af9822 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:45.487361 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 28/60] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:58:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:45.493859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed22ab-111c-4bc9-b64a-81d6feca1140 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:45.532616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:45.605445 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed22ab-111c-4bc9-b64a-81d6feca1140 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:45.639774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da12e8c-8f50-4a82-aef2-a59c87027454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:45.640439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da12e8c-8f50-4a82-aef2-a59c87027454 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:45.640723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da12e8c-8f50-4a82-aef2-a59c87027454 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:45.640996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da12e8c-8f50-4a82-aef2-a59c87027454 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:45.641714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da12e8c-8f50-4a82-aef2-a59c87027454 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:45.643197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da12e8c-8f50-4a82-aef2-a59c87027454 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:45.675128 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 33/61] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:58:45 2022] POST /identity/v3/users => generated 275 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 09:58:48.690414 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c2e02-49d9-46a1-ade6-09d21356a00a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:48.690959 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c2e02-49d9-46a1-ade6-09d21356a00a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:48.690959 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c2e02-49d9-46a1-ade6-09d21356a00a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:48.692663 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 29/62] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:48.697704 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b2f7-053c-4ce0-8186-16490e7b3828 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:48.698565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b2f7-053c-4ce0-8186-16490e7b3828 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:48.699096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b2f7-053c-4ce0-8186-16490e7b3828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:48.724015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-1c37b2f7-053c-4ce0-8186-16490e7b3828 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:48.725569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1c37b2f7-053c-4ce0-8186-16490e7b3828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:48.802174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cdaff-7474-4219-8098-2efc763ed6fe None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:48.802567 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cdaff-7474-4219-8098-2efc763ed6fe None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:48.802804 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cdaff-7474-4219-8098-2efc763ed6fe None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:48.804367 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 30/63] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 09:58:48.818243 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2cbbf-f571-4f77-9126-c84d254fc579 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:48.818529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2cbbf-f571-4f77-9126-c84d254fc579 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:48.818808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2cbbf-f571-4f77-9126-c84d254fc579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:48.837328 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 34/64] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:48.841141 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-afa2cbbf-f571-4f77-9126-c84d254fc579 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:48.847092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-afa2cbbf-f571-4f77-9126-c84d254fc579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:48.895330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37b2f7-053c-4ce0-8186-16490e7b3828 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:48.930602 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:48.931784 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 31/65] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:58:48.955231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2cbbf-f571-4f77-9126-c84d254fc579 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:48.990702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:49.058252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37b2f7-053c-4ce0-8186-16490e7b3828 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:49.058445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2cbbf-f571-4f77-9126-c84d254fc579 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:49.090104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70565d61-fcc3-4e7a-ab73-68c7ebbd30b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:49.091279 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70565d61-fcc3-4e7a-ab73-68c7ebbd30b8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.091732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70565d61-fcc3-4e7a-ab73-68c7ebbd30b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.092257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70565d61-fcc3-4e7a-ab73-68c7ebbd30b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.103153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70565d61-fcc3-4e7a-ab73-68c7ebbd30b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-70565d61-fcc3-4e7a-ab73-68c7ebbd30b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:49.111319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:49.112152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:49.113030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:58:49.115573 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 32/66] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:58:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:58:49.119777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612ce4b-9f58-4b24-8f6a-ad110acf445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:49.120779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ce4b-9f58-4b24-8f6a-ad110acf445a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.121226 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ce4b-9f58-4b24-8f6a-ad110acf445a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.121663 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ce4b-9f58-4b24-8f6a-ad110acf445a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.132023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70565d61-fcc3-4e7a-ab73-68c7ebbd30b8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:49.137300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612ce4b-9f58-4b24-8f6a-ad110acf445a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-c612ce4b-9f58-4b24-8f6a-ad110acf445a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:49.145379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:49.146388 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:58:49.147153 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 35/67] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:49.148226 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd52eb-ea35-4a97-acc2-f3ca82de105e None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.148876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd52eb-ea35-4a97-acc2-f3ca82de105e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.149354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd52eb-ea35-4a97-acc2-f3ca82de105e None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.150914 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 36/68] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:49.155919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd52eb-ea35-4a97-acc2-f3ca82de105e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:49.227141 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:49.247361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70565d61-fcc3-4e7a-ab73-68c7ebbd30b8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:49.254572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:49.266297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd52eb-ea35-4a97-acc2-f3ca82de105e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:49.284853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed7f5e-8665-48ff-a51e-7424a5bbdd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:49.285879 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed7f5e-8665-48ff-a51e-7424a5bbdd5e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.286379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed7f5e-8665-48ff-a51e-7424a5bbdd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.286902 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed7f5e-8665-48ff-a51e-7424a5bbdd5e None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.287884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed7f5e-8665-48ff-a51e-7424a5bbdd5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:49.288378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed7f5e-8665-48ff-a51e-7424a5bbdd5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:49.289952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed7f5e-8665-48ff-a51e-7424a5bbdd5e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:49.299174 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 33/69] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:49.300745 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b51cc-55b3-4408-a625-64845038dd6c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.301226 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b51cc-55b3-4408-a625-64845038dd6c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.301808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b51cc-55b3-4408-a625-64845038dd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.305308 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa26e53-92d5-4e94-a4b0-e74d4107ed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:49.306026 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26e53-92d5-4e94-a4b0-e74d4107ed5e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.306385 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26e53-92d5-4e94-a4b0-e74d4107ed5e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.306671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26e53-92d5-4e94-a4b0-e74d4107ed5e None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.307477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa26e53-92d5-4e94-a4b0-e74d4107ed5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:49.307795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa26e53-92d5-4e94-a4b0-e74d4107ed5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:49.310174 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa26e53-92d5-4e94-a4b0-e74d4107ed5e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:49.319499 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 37/70] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:58:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:49.320522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa26e53-92d5-4e94-a4b0-e74d4107ed5e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:49.321651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f73b51cc-55b3-4408-a625-64845038dd6c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:49.323055 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f73b51cc-55b3-4408-a625-64845038dd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:49.391443 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 34/71] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:49.392765 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b51cc-55b3-4408-a625-64845038dd6c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:49.442071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa26e53-92d5-4e94-a4b0-e74d4107ed5e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:49.470942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:49.479157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355dca2-d370-47e6-8d2e-c4a4f2739e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:49.479981 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355dca2-d370-47e6-8d2e-c4a4f2739e9b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.480265 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355dca2-d370-47e6-8d2e-c4a4f2739e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.480542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355dca2-d370-47e6-8d2e-c4a4f2739e9b None admin] PATH_INFO: `/v3/users/c042e1f397444e8991d3f0b7bfb28908` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.482007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355dca2-d370-47e6-8d2e-c4a4f2739e9b None admin] RBAC: Authorizing `identity:get_user(user_id=c042e1f397444e8991d3f0b7bfb28908)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:49.483521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355dca2-d370-47e6-8d2e-c4a4f2739e9b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:49.485173 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 38/72] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:49 2022] GET /identity/v3/users/c042e1f397444e8991d3f0b7bfb28908 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:49.485948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355dca2-d370-47e6-8d2e-c4a4f2739e9b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:49.496312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:49.496858 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b51cc-55b3-4408-a625-64845038dd6c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:49.529484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54e8bb-2168-478e-989c-161686bf73e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:49.530407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54e8bb-2168-478e-989c-161686bf73e1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.530691 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54e8bb-2168-478e-989c-161686bf73e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.530963 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54e8bb-2168-478e-989c-161686bf73e1 None admin] PATH_INFO: `/v3/users/e94fb5eec4f149e686cb01446e186584` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.543890 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54e8bb-2168-478e-989c-161686bf73e1 None admin] RBAC: Authorizing `identity:get_user(user_id=e94fb5eec4f149e686cb01446e186584)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:49.545359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54e8bb-2168-478e-989c-161686bf73e1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:49.547399 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 35/73] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:49 2022] GET /identity/v3/users/e94fb5eec4f149e686cb01446e186584 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:49.552219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f297a-c610-4ee2-bfb8-3a0b58c081f7 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.552560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f297a-c610-4ee2-bfb8-3a0b58c081f7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.552844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f297a-c610-4ee2-bfb8-3a0b58c081f7 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.554330 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 36/74] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:49.555422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900dcc-c3db-4c8f-acda-f6b8f7d2d06b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.555717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900dcc-c3db-4c8f-acda-f6b8f7d2d06b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.555997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900dcc-c3db-4c8f-acda-f6b8f7d2d06b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.557238 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 37/75] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:49.557937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1900dcc-c3db-4c8f-acda-f6b8f7d2d06b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:49.586821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355dca2-d370-47e6-8d2e-c4a4f2739e9b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:49.619150 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9b062-2ee8-4c75-94a9-9bbb7d3f1eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:49.620012 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9b062-2ee8-4c75-94a9-9bbb7d3f1eda None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.620301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9b062-2ee8-4c75-94a9-9bbb7d3f1eda None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.620587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9b062-2ee8-4c75-94a9-9bbb7d3f1eda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.630936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9b062-2ee8-4c75-94a9-9bbb7d3f1eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-44e9b062-2ee8-4c75-94a9-9bbb7d3f1eda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:49.638982 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:49.639832 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:58:49.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 39/76] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:58:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:49.641261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9b062-2ee8-4c75-94a9-9bbb7d3f1eda None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:49.666594 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1900dcc-c3db-4c8f-acda-f6b8f7d2d06b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:49.695312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:49.700505 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262d927-b040-4ca3-8348-5d94b97c8c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:49.701182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262d927-b040-4ca3-8348-5d94b97c8c9a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.701470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262d927-b040-4ca3-8348-5d94b97c8c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.701750 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262d927-b040-4ca3-8348-5d94b97c8c9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.703548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:49.738121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9b062-2ee8-4c75-94a9-9bbb7d3f1eda None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:49.771799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a53940-c642-4af0-aecc-75b421400098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:49.772469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a53940-c642-4af0-aecc-75b421400098 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.772588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262d927-b040-4ca3-8348-5d94b97c8c9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:49.772753 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a53940-c642-4af0-aecc-75b421400098 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.773029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a53940-c642-4af0-aecc-75b421400098 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.774125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262d927-b040-4ca3-8348-5d94b97c8c9a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:49.842922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a53940-c642-4af0-aecc-75b421400098 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:49.844449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a53940-c642-4af0-aecc-75b421400098 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:49.876543 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 38/77] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:58:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:58:49.876543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d1c8-cffd-47bb-91a9-f138450a527d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.876543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d1c8-cffd-47bb-91a9-f138450a527d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.876543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d1c8-cffd-47bb-91a9-f138450a527d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.890971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b9a8d1c8-cffd-47bb-91a9-f138450a527d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:49.892199 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b9a8d1c8-cffd-47bb-91a9-f138450a527d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:49.910627 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:49.917610 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:49.944897 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 40/78] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:58:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:58:49.946322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1692d-bd0e-47e0-90d1-3541e336b380 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:49.946495 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1692d-bd0e-47e0-90d1-3541e336b380 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:49.946753 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1692d-bd0e-47e0-90d1-3541e336b380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:49.960934 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 39/79] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:49.961601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8d1c8-cffd-47bb-91a9-f138450a527d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:49.963994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-6be1692d-bd0e-47e0-90d1-3541e336b380 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:49.965174 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6be1692d-bd0e-47e0-90d1-3541e336b380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:50.037783 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 41/80] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:50.038668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1692d-bd0e-47e0-90d1-3541e336b380 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:50.068194 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8d1c8-cffd-47bb-91a9-f138450a527d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:50.101506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fbce8-59f9-45b0-a659-3fd0d2d2f3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:50.102183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fbce8-59f9-45b0-a659-3fd0d2d2f3bf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:50.102513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fbce8-59f9-45b0-a659-3fd0d2d2f3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:50.102793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fbce8-59f9-45b0-a659-3fd0d2d2f3bf None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:50.103557 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fbce8-59f9-45b0-a659-3fd0d2d2f3bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:50.103876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fbce8-59f9-45b0-a659-3fd0d2d2f3bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:50.105276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fbce8-59f9-45b0-a659-3fd0d2d2f3bf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:50.113089 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 40/81] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:58:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:50.113891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fbce8-59f9-45b0-a659-3fd0d2d2f3bf None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:50.134430 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:50.138402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:50.141222 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1692d-bd0e-47e0-90d1-3541e336b380 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:50.173545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51cc0d-2253-4660-a0e3-ea187c95e6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:50.174465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51cc0d-2253-4660-a0e3-ea187c95e6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:50.174752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51cc0d-2253-4660-a0e3-ea187c95e6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:50.175031 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51cc0d-2253-4660-a0e3-ea187c95e6d7 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:50.176247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51cc0d-2253-4660-a0e3-ea187c95e6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:50.177832 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51cc0d-2253-4660-a0e3-ea187c95e6d7 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:50.179557 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 42/82] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:58:50 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:50.183051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51cc0d-2253-4660-a0e3-ea187c95e6d7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:50.213381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fbce8-59f9-45b0-a659-3fd0d2d2f3bf None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:50.245199 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d706bc-0823-43e7-9e8f-8de24c84f576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:50.246061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d706bc-0823-43e7-9e8f-8de24c84f576 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:50.246378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d706bc-0823-43e7-9e8f-8de24c84f576 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:50.246666 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d706bc-0823-43e7-9e8f-8de24c84f576 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:50.247856 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d706bc-0823-43e7-9e8f-8de24c84f576 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:50.249307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d706bc-0823-43e7-9e8f-8de24c84f576 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:50.251027 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 41/83] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:58:50 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:50.255312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d706bc-0823-43e7-9e8f-8de24c84f576 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:50.278986 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51cc0d-2253-4660-a0e3-ea187c95e6d7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:50.311918 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f585fd-6dcb-4080-a177-c3bff814a224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:50.312649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f585fd-6dcb-4080-a177-c3bff814a224 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:50.312932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f585fd-6dcb-4080-a177-c3bff814a224 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:50.313214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f585fd-6dcb-4080-a177-c3bff814a224 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:50.323396 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f585fd-6dcb-4080-a177-c3bff814a224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-d1f585fd-6dcb-4080-a177-c3bff814a224 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:50.331141 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:50.332320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:58:50.333324 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 43/84] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:50.334199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f585fd-6dcb-4080-a177-c3bff814a224 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:50.344573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:50.353735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d706bc-0823-43e7-9e8f-8de24c84f576 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:50.358674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:50.388158 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03185317-55d8-46a1-9ee1-016527a2c0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:50.389009 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03185317-55d8-46a1-9ee1-016527a2c0ee None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:50.389298 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03185317-55d8-46a1-9ee1-016527a2c0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:50.389577 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03185317-55d8-46a1-9ee1-016527a2c0ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:50.390793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03185317-55d8-46a1-9ee1-016527a2c0ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:50.392276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03185317-55d8-46a1-9ee1-016527a2c0ee None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:50.393895 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 42/85] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:58:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:50.394690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03185317-55d8-46a1-9ee1-016527a2c0ee None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:50.436820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f585fd-6dcb-4080-a177-c3bff814a224 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:50.470872 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b5591-221a-4c80-bf5a-1a0ee0a44273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:50.471773 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b5591-221a-4c80-bf5a-1a0ee0a44273 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:50.472050 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b5591-221a-4c80-bf5a-1a0ee0a44273 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:50.472325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b5591-221a-4c80-bf5a-1a0ee0a44273 None admin] PATH_INFO: `/v3/users/e94fb5eec4f149e686cb01446e186584` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:50.473516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b5591-221a-4c80-bf5a-1a0ee0a44273 None admin] RBAC: Authorizing `identity:get_user(user_id=e94fb5eec4f149e686cb01446e186584)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:50.475125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b5591-221a-4c80-bf5a-1a0ee0a44273 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:50.476814 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 44/86] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:50 2022] GET /identity/v3/users/e94fb5eec4f149e686cb01446e186584 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:50.479383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b5591-221a-4c80-bf5a-1a0ee0a44273 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:50.494054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03185317-55d8-46a1-9ee1-016527a2c0ee None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:50.527290 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf154dd-41c3-4215-9b29-d3a7479e48c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:50.527984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf154dd-41c3-4215-9b29-d3a7479e48c4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:50.528276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf154dd-41c3-4215-9b29-d3a7479e48c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:50.528558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf154dd-41c3-4215-9b29-d3a7479e48c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:50.529404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf154dd-41c3-4215-9b29-d3a7479e48c4 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4, user.id=c042e1f397444e8991d3f0b7bfb28908) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:50.529719 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf154dd-41c3-4215-9b29-d3a7479e48c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:50.531296 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf154dd-41c3-4215-9b29-d3a7479e48c4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:50.536656 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 43/87] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:58:50 2022] GET /identity/v3/role_assignments?user.id=c042e1f397444e8991d3f0b7bfb28908&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=c184a177010144429f495f3290d28221 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:50.537485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf154dd-41c3-4215-9b29-d3a7479e48c4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:50.567207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:50.573742 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:50.584536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b5591-221a-4c80-bf5a-1a0ee0a44273 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:50.623887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2528d5-45ba-4afb-8780-cbb346f87823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:50.624620 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2528d5-45ba-4afb-8780-cbb346f87823 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:50.624910 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2528d5-45ba-4afb-8780-cbb346f87823 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:50.625189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2528d5-45ba-4afb-8780-cbb346f87823 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:50.626050 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2528d5-45ba-4afb-8780-cbb346f87823 None admin] RBAC: Adding query filter params (role.id=b10cd0627cc84a9c92dba333901e60e1, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4, user.id=e94fb5eec4f149e686cb01446e186584) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:50.627032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2528d5-45ba-4afb-8780-cbb346f87823 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:50.629037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2528d5-45ba-4afb-8780-cbb346f87823 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:50.638878 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 45/88] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:58:50 2022] GET /identity/v3/role_assignments?user.id=e94fb5eec4f149e686cb01446e186584&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:50.639555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2528d5-45ba-4afb-8780-cbb346f87823 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:50.664517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf154dd-41c3-4215-9b29-d3a7479e48c4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:50.715996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72092c1b-77e7-4bef-bb24-afc09384f48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:50.716785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72092c1b-77e7-4bef-bb24-afc09384f48f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:50.717190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72092c1b-77e7-4bef-bb24-afc09384f48f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:50.717477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72092c1b-77e7-4bef-bb24-afc09384f48f None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:50.718298 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72092c1b-77e7-4bef-bb24-afc09384f48f None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:50.718613 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72092c1b-77e7-4bef-bb24-afc09384f48f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:50.720184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72092c1b-77e7-4bef-bb24-afc09384f48f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:50.734269 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 44/89] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:58:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:50.734269 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72092c1b-77e7-4bef-bb24-afc09384f48f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:50.786087 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2528d5-45ba-4afb-8780-cbb346f87823 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:50.826596 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8917309-b50e-42c0-b1d7-217620f761f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:50.827506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8917309-b50e-42c0-b1d7-217620f761f4 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:50.827939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8917309-b50e-42c0-b1d7-217620f761f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:50.828511 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8917309-b50e-42c0-b1d7-217620f761f4 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:50.829541 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8917309-b50e-42c0-b1d7-217620f761f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:50.831042 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8917309-b50e-42c0-b1d7-217620f761f4 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:50.852196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:50.863404 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 46/90] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:58:50 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:58:50.864623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8917309-b50e-42c0-b1d7-217620f761f4 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:50.881473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72092c1b-77e7-4bef-bb24-afc09384f48f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:50.884757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:50.991891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8917309-b50e-42c0-b1d7-217620f761f4 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:51.011644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e9c2-a8ff-4af7-8d30-600d3d290d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:51.012324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e9c2-a8ff-4af7-8d30-600d3d290d1d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:51.012611 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e9c2-a8ff-4af7-8d30-600d3d290d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:51.012891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e9c2-a8ff-4af7-8d30-600d3d290d1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:51.032336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10a78c-aee8-400e-90aa-414f53765c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:51.033552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10a78c-aee8-400e-90aa-414f53765c49 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:51.034016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10a78c-aee8-400e-90aa-414f53765c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:51.034522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10a78c-aee8-400e-90aa-414f53765c49 None admin] PATH_INFO: `/v3/users/e94fb5eec4f149e686cb01446e186584` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:51.036551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10a78c-aee8-400e-90aa-414f53765c49 None admin] RBAC: Authorizing `identity:get_user(user_id=e94fb5eec4f149e686cb01446e186584)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:51.038355 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10a78c-aee8-400e-90aa-414f53765c49 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:51.040656 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 47/91] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:50 2022] GET /identity/v3/users/e94fb5eec4f149e686cb01446e186584 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:51.047031 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10a78c-aee8-400e-90aa-414f53765c49 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:51.149241 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280e9c2-a8ff-4af7-8d30-600d3d290d1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:51.151091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280e9c2-a8ff-4af7-8d30-600d3d290d1d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:51.159404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:51.180471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10a78c-aee8-400e-90aa-414f53765c49 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:51.219588 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ee698-31aa-4ad6-a1ba-368e8295ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:51.220081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee698-31aa-4ad6-a1ba-368e8295ee40 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:51.220405 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee698-31aa-4ad6-a1ba-368e8295ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:51.220701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee698-31aa-4ad6-a1ba-368e8295ee40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:51.249401 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:51.272379 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 45/92] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:58:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:58:51.278107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e9c2-a8ff-4af7-8d30-600d3d290d1d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:51.327989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ee698-31aa-4ad6-a1ba-368e8295ee40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:51.330404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ee698-31aa-4ad6-a1ba-368e8295ee40 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:51.412587 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e9c2-a8ff-4af7-8d30-600d3d290d1d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:51.437219 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:51.455127 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3629ec-8a8d-4022-a946-12dc7d1fc925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:51.456376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3629ec-8a8d-4022-a946-12dc7d1fc925 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:51.456855 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3629ec-8a8d-4022-a946-12dc7d1fc925 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:51.457631 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3629ec-8a8d-4022-a946-12dc7d1fc925 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:51.459182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3629ec-8a8d-4022-a946-12dc7d1fc925 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:51.460870 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3629ec-8a8d-4022-a946-12dc7d1fc925 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:51.462734 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 48/93] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:58:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:58:51.467316 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 46/94] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:58:51 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:51.468635 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ee698-31aa-4ad6-a1ba-368e8295ee40 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:51.475281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3629ec-8a8d-4022-a946-12dc7d1fc925 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:51.560708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:51.591633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ee698-31aa-4ad6-a1ba-368e8295ee40 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:51.617730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3629ec-8a8d-4022-a946-12dc7d1fc925 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:51.642529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d70a79-3fa7-4f2f-bdd6-6a1282e4b51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:51.643469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d70a79-3fa7-4f2f-bdd6-6a1282e4b51a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:51.643753 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d70a79-3fa7-4f2f-bdd6-6a1282e4b51a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:51.644026 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d70a79-3fa7-4f2f-bdd6-6a1282e4b51a None admin] PATH_INFO: `/v3/projects/5d649fc3e290424ca5d67aa5529a2393` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:51.646658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d70a79-3fa7-4f2f-bdd6-6a1282e4b51a None admin] RBAC: Authorizing `identity:get_project(project_id=5d649fc3e290424ca5d67aa5529a2393)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:51.648145 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d70a79-3fa7-4f2f-bdd6-6a1282e4b51a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:51.650322 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 49/95] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:58:51 2022] GET /identity/v3/projects/5d649fc3e290424ca5d67aa5529a2393 => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:51.656487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d70a79-3fa7-4f2f-bdd6-6a1282e4b51a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:51.693532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e93e75-2c3e-4326-81e4-033ac4398c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:51.694556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e93e75-2c3e-4326-81e4-033ac4398c4d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:51.694845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e93e75-2c3e-4326-81e4-033ac4398c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:51.695187 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e93e75-2c3e-4326-81e4-033ac4398c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:51.696038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e93e75-2c3e-4326-81e4-033ac4398c4d None admin] RBAC: Adding query filter params (role.id=70d38d0a968a4e9585c5873d24cfbbb1, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4, user.id=e94fb5eec4f149e686cb01446e186584) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:51.696408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e93e75-2c3e-4326-81e4-033ac4398c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:51.697867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e93e75-2c3e-4326-81e4-033ac4398c4d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:51.708674 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 47/96] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:58:51 2022] GET /identity/v3/role_assignments?user.id=e94fb5eec4f149e686cb01446e186584&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=70d38d0a968a4e9585c5873d24cfbbb1 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:51.746899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:51.828547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d70a79-3fa7-4f2f-bdd6-6a1282e4b51a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:51.888001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991070d-d8c8-4ad7-a67e-f638226df8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:51.889108 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991070d-d8c8-4ad7-a67e-f638226df8bc None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:51.889647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991070d-d8c8-4ad7-a67e-f638226df8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:51.889934 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991070d-d8c8-4ad7-a67e-f638226df8bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:51.890873 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991070d-d8c8-4ad7-a67e-f638226df8bc None admin] RBAC: Adding query filter params (role.id=b10cd0627cc84a9c92dba333901e60e1, scope.project.id=5d649fc3e290424ca5d67aa5529a2393, user.id=e94fb5eec4f149e686cb01446e186584) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:51.891194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991070d-d8c8-4ad7-a67e-f638226df8bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:51.892791 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991070d-d8c8-4ad7-a67e-f638226df8bc None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:51.903145 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 50/97] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:58:51 2022] GET /identity/v3/role_assignments?user.id=e94fb5eec4f149e686cb01446e186584&scope.project.id=5d649fc3e290424ca5d67aa5529a2393&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:52.861437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed9df9-e1d2-4978-a4f8-23a6e65f384a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:52.862043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed9df9-e1d2-4978-a4f8-23a6e65f384a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:52.862043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed9df9-e1d2-4978-a4f8-23a6e65f384a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:52.863633 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 48/98] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:52.869535 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d5646-bd6d-4bf2-b1dd-a3c5d16707e9 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:52.870457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d5646-bd6d-4bf2-b1dd-a3c5d16707e9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:52.870963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d5646-bd6d-4bf2-b1dd-a3c5d16707e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:52.888487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-474d5646-bd6d-4bf2-b1dd-a3c5d16707e9 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:52.890021 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-474d5646-bd6d-4bf2-b1dd-a3c5d16707e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:52.962363 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 51/99] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:53.011830 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed9df9-e1d2-4978-a4f8-23a6e65f384a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:53.124388 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed9df9-e1d2-4978-a4f8-23a6e65f384a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:53.146653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:53.166072 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32617624-2bd3-4303-959f-aa740f8b800e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:53.167297 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32617624-2bd3-4303-959f-aa740f8b800e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:53.167788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32617624-2bd3-4303-959f-aa740f8b800e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:53.168258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32617624-2bd3-4303-959f-aa740f8b800e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:53.181721 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32617624-2bd3-4303-959f-aa740f8b800e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-32617624-2bd3-4303-959f-aa740f8b800e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:53.189832 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:53.190738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:53.191525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:58:53.193987 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 49/100] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:58:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:53.200237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d5646-bd6d-4bf2-b1dd-a3c5d16707e9 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:53.255277 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:53.328500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d5646-bd6d-4bf2-b1dd-a3c5d16707e9 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:53.369393 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f1a85-6609-4c5e-acbc-6e28bc83d498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:53.370085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f1a85-6609-4c5e-acbc-6e28bc83d498 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:53.370411 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f1a85-6609-4c5e-acbc-6e28bc83d498 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:53.370698 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f1a85-6609-4c5e-acbc-6e28bc83d498 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:53.371464 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f1a85-6609-4c5e-acbc-6e28bc83d498 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:53.371783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f1a85-6609-4c5e-acbc-6e28bc83d498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:53.373199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f1a85-6609-4c5e-acbc-6e28bc83d498 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:53.382082 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 52/101] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:53.388118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32617624-2bd3-4303-959f-aa740f8b800e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:53.466930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561eed59-3149-479b-834e-4076b28f32a9 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:53.467256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561eed59-3149-479b-834e-4076b28f32a9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:53.467505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561eed59-3149-479b-834e-4076b28f32a9 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:53.469068 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 53/102] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 09:58:53.479596 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61ef0-7c92-4154-8dd9-45823d8bac90 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:53.479890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61ef0-7c92-4154-8dd9-45823d8bac90 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:53.480179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61ef0-7c92-4154-8dd9-45823d8bac90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:53.503043 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-fdf61ef0-7c92-4154-8dd9-45823d8bac90 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:53.504295 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-fdf61ef0-7c92-4154-8dd9-45823d8bac90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:53.566878 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32617624-2bd3-4303-959f-aa740f8b800e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:53.635444 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ffe4f-6cee-4eda-96e3-b186e9a5c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:53.636332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ffe4f-6cee-4eda-96e3-b186e9a5c996 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:53.636622 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ffe4f-6cee-4eda-96e3-b186e9a5c996 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:53.636904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ffe4f-6cee-4eda-96e3-b186e9a5c996 None admin] PATH_INFO: `/v3/users/c042e1f397444e8991d3f0b7bfb28908` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:53.638776 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ffe4f-6cee-4eda-96e3-b186e9a5c996 None admin] RBAC: Authorizing `identity:get_user(user_id=c042e1f397444e8991d3f0b7bfb28908)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:53.640248 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ffe4f-6cee-4eda-96e3-b186e9a5c996 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:53.642072 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 50/103] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:53 2022] GET /identity/v3/users/c042e1f397444e8991d3f0b7bfb28908 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:53.649175 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 54/104] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:58:53.649317 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ffe4f-6cee-4eda-96e3-b186e9a5c996 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:53.683093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf61ef0-7c92-4154-8dd9-45823d8bac90 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:53.725060 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:53.734561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:53.811319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ffe4f-6cee-4eda-96e3-b186e9a5c996 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:53.846514 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf61ef0-7c92-4154-8dd9-45823d8bac90 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:53.874991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033a542-2443-414f-a87e-97c09608c277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:53.875870 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033a542-2443-414f-a87e-97c09608c277 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:53.876158 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033a542-2443-414f-a87e-97c09608c277 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:53.876433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033a542-2443-414f-a87e-97c09608c277 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:53.878415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033a542-2443-414f-a87e-97c09608c277 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:53.879875 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033a542-2443-414f-a87e-97c09608c277 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:53.885825 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 51/105] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:58:53 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:53.892851 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033a542-2443-414f-a87e-97c09608c277 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:53.918518 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be296c14-f20c-4500-9918-cfc4f99a13c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:53.919336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be296c14-f20c-4500-9918-cfc4f99a13c7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:53.919618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be296c14-f20c-4500-9918-cfc4f99a13c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:53.919894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be296c14-f20c-4500-9918-cfc4f99a13c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:53.944121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be296c14-f20c-4500-9918-cfc4f99a13c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-be296c14-f20c-4500-9918-cfc4f99a13c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:53.952407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:53.953295 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:58:53.954082 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:58:53.958884 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 55/106] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:58:53.966294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be296c14-f20c-4500-9918-cfc4f99a13c7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:54.152097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:54.164333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be296c14-f20c-4500-9918-cfc4f99a13c7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:54.181070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033a542-2443-414f-a87e-97c09608c277 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:54.211370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:54.219026 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9939cd-e2a8-40e2-ad17-02874b6b60e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:54.219798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9939cd-e2a8-40e2-ad17-02874b6b60e9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.220097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9939cd-e2a8-40e2-ad17-02874b6b60e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.220432 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9939cd-e2a8-40e2-ad17-02874b6b60e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.221272 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9939cd-e2a8-40e2-ad17-02874b6b60e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:54.221606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9939cd-e2a8-40e2-ad17-02874b6b60e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:54.223168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9939cd-e2a8-40e2-ad17-02874b6b60e9 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:54.236990 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 56/107] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:58:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:54.237989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9939cd-e2a8-40e2-ad17-02874b6b60e9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:54.270996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e2622-0e64-4133-bb68-cdb7ac9a91c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:54.272054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e2622-0e64-4133-bb68-cdb7ac9a91c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.272379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e2622-0e64-4133-bb68-cdb7ac9a91c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.272634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e2622-0e64-4133-bb68-cdb7ac9a91c1 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/users/c042e1f397444e8991d3f0b7bfb28908/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.274785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e2622-0e64-4133-bb68-cdb7ac9a91c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d72a2d59fc146d3afa2a17c4f4460f4, user_id=c042e1f397444e8991d3f0b7bfb28908, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:54.276425 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e2622-0e64-4133-bb68-cdb7ac9a91c1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:54.300809 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 52/108] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:58:53 2022] PUT /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/users/c042e1f397444e8991d3f0b7bfb28908/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:58:54.409660 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f63a63-4742-4079-bc31-e4566014977b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.409961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f63a63-4742-4079-bc31-e4566014977b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.410153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f63a63-4742-4079-bc31-e4566014977b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.411656 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 53/109] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 09:58:54.412191 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9939cd-e2a8-40e2-ad17-02874b6b60e9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:54.417533 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744fef8-71e7-4dcd-8378-f6357af1406b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.417936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744fef8-71e7-4dcd-8378-f6357af1406b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.418361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744fef8-71e7-4dcd-8378-f6357af1406b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.458240 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5744fef8-71e7-4dcd-8378-f6357af1406b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:54.459582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5744fef8-71e7-4dcd-8378-f6357af1406b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:54.475117 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa8411-fd98-4ead-b8b7-0dd9084e7a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:54.476087 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa8411-fd98-4ead-b8b7-0dd9084e7a05 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.476373 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa8411-fd98-4ead-b8b7-0dd9084e7a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.476654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa8411-fd98-4ead-b8b7-0dd9084e7a05 None admin] PATH_INFO: `/v3/users/e94fb5eec4f149e686cb01446e186584` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.477995 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa8411-fd98-4ead-b8b7-0dd9084e7a05 None admin] RBAC: Authorizing `identity:get_user(user_id=e94fb5eec4f149e686cb01446e186584)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:54.479628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa8411-fd98-4ead-b8b7-0dd9084e7a05 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:54.481747 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 57/110] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:54 2022] GET /identity/v3/users/e94fb5eec4f149e686cb01446e186584 => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:54.487651 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa8411-fd98-4ead-b8b7-0dd9084e7a05 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:54.526539 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:54.619252 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 54/111] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:58:54.622104 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa8411-fd98-4ead-b8b7-0dd9084e7a05 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:54.652675 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744fef8-71e7-4dcd-8378-f6357af1406b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:54.660051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fad128-2eb4-455d-8ebf-9b8015e6f2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:54.660905 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fad128-2eb4-455d-8ebf-9b8015e6f2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.661229 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fad128-2eb4-455d-8ebf-9b8015e6f2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.661516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fad128-2eb4-455d-8ebf-9b8015e6f2d1 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.663891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fad128-2eb4-455d-8ebf-9b8015e6f2d1 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:54.665409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fad128-2eb4-455d-8ebf-9b8015e6f2d1 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:54.667568 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 58/112] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:58:54 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:54.674759 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fad128-2eb4-455d-8ebf-9b8015e6f2d1 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:54.794712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:54.803497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fad128-2eb4-455d-8ebf-9b8015e6f2d1 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:54.867978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744fef8-71e7-4dcd-8378-f6357af1406b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:54.870806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:54.876767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f60e28-d737-4017-abdd-55ddeb5154df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:54.877846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f60e28-d737-4017-abdd-55ddeb5154df None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.878001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f60e28-d737-4017-abdd-55ddeb5154df None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.878325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f60e28-d737-4017-abdd-55ddeb5154df None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/users/e94fb5eec4f149e686cb01446e186584/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.880878 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f60e28-d737-4017-abdd-55ddeb5154df None admin] RBAC: Authorizing `identity:create_grant(project_id=1d72a2d59fc146d3afa2a17c4f4460f4, user_id=e94fb5eec4f149e686cb01446e186584, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:54.882472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f60e28-d737-4017-abdd-55ddeb5154df None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:54.900682 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 59/113] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:58:54 2022] PUT /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/users/e94fb5eec4f149e686cb01446e186584/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 09:58:54.902396 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e75da-ebcb-48b9-b164-a0f3ad6dd5ff None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.902814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e75da-ebcb-48b9-b164-a0f3ad6dd5ff None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.903219 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e75da-ebcb-48b9-b164-a0f3ad6dd5ff None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.904754 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 60/114] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:54.909472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4e03-191d-4cd8-9291-edbc70046cfd None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.909472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4e03-191d-4cd8-9291-edbc70046cfd None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.909472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4e03-191d-4cd8-9291-edbc70046cfd None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.909472 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 61/115] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:54.913567 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5db91-8bf2-4dc5-9691-5f71472e900c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:54.914111 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab222-179c-47a5-98fe-baa4686c5a42 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.914279 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab222-179c-47a5-98fe-baa4686c5a42 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.914621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab222-179c-47a5-98fe-baa4686c5a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.915742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5db91-8bf2-4dc5-9691-5f71472e900c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.916034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5db91-8bf2-4dc5-9691-5f71472e900c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.916335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5db91-8bf2-4dc5-9691-5f71472e900c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.933804 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5db91-8bf2-4dc5-9691-5f71472e900c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:54.936196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-521ab222-179c-47a5-98fe-baa4686c5a42 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:54.937966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-521ab222-179c-47a5-98fe-baa4686c5a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-bad5db91-8bf2-4dc5-9691-5f71472e900c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:54.943795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:54.944750 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:58:54.945602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:58:54.951696 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 55/116] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:58:54.951696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b5441-845d-49d7-8272-075b7e0b973f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:54.951696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b5441-845d-49d7-8272-075b7e0b973f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:54.953275 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b5441-845d-49d7-8272-075b7e0b973f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:54.991531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f13b5441-845d-49d7-8272-075b7e0b973f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:54.993519 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f13b5441-845d-49d7-8272-075b7e0b973f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:55.102573 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 62/117] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:58:55.104055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ab222-179c-47a5-98fe-baa4686c5a42 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:55.144287 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 56/118] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:55.146345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b5441-845d-49d7-8272-075b7e0b973f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:55.197369 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:55.219018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ab222-179c-47a5-98fe-baa4686c5a42 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:55.262264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f0b62-00c6-42ae-8151-9bb554b43590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:55.262264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0b62-00c6-42ae-8151-9bb554b43590 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:55.262264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0b62-00c6-42ae-8151-9bb554b43590 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:55.262264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0b62-00c6-42ae-8151-9bb554b43590 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:55.262686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f0b62-00c6-42ae-8151-9bb554b43590 None admin] RBAC: Adding query filter params (name=member) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:55.262686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f0b62-00c6-42ae-8151-9bb554b43590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:55.262686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f0b62-00c6-42ae-8151-9bb554b43590 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:55.284355 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b5441-845d-49d7-8272-075b7e0b973f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:55.286099 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:55.287261 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 63/119] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:58:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:55.288939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8078d0-5582-4af0-9739-ae8038c11746 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:55.289414 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8078d0-5582-4af0-9739-ae8038c11746 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:55.289710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8078d0-5582-4af0-9739-ae8038c11746 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:55.291194 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 64/120] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:55.291949 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8078d0-5582-4af0-9739-ae8038c11746 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:55.324833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9a57b-4d32-4ec7-bc31-2f73bf4a2d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:55.325632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9a57b-4d32-4ec7-bc31-2f73bf4a2d32 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:55.325922 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9a57b-4d32-4ec7-bc31-2f73bf4a2d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:55.326204 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9a57b-4d32-4ec7-bc31-2f73bf4a2d32 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:55.337368 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9a57b-4d32-4ec7-bc31-2f73bf4a2d32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6ed9a57b-4d32-4ec7-bc31-2f73bf4a2d32 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:55.345187 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:55.346078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:58:55.346868 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:58:55.350300 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 57/121] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:55.351178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9a57b-4d32-4ec7-bc31-2f73bf4a2d32 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:55.407323 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8078d0-5582-4af0-9739-ae8038c11746 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:55.454002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b5d76-44f7-41dc-b773-e3d3386d4e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:55.455029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5d76-44f7-41dc-b773-e3d3386d4e32 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:55.455708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5d76-44f7-41dc-b773-e3d3386d4e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:55.456002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5d76-44f7-41dc-b773-e3d3386d4e32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:55.457773 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b5d76-44f7-41dc-b773-e3d3386d4e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:55.460240 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b5d76-44f7-41dc-b773-e3d3386d4e32 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:55.462568 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 65/122] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:58:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:55.464018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb465f-0ac3-428c-9583-656345f444f3 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:55.464304 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb465f-0ac3-428c-9583-656345f444f3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:55.464583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb465f-0ac3-428c-9583-656345f444f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:55.464935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9a57b-4d32-4ec7-bc31-2f73bf4a2d32 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:55.485779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-fdbb465f-0ac3-428c-9583-656345f444f3 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:55.487033 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-fdbb465f-0ac3-428c-9583-656345f444f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:55.499782 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:55.507005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cdcc7-7efd-4835-9b03-1a517c577b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:55.508123 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cdcc7-7efd-4835-9b03-1a517c577b09 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:55.508706 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cdcc7-7efd-4835-9b03-1a517c577b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:55.509050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cdcc7-7efd-4835-9b03-1a517c577b09 None admin] PATH_INFO: `/v3/users/e94fb5eec4f149e686cb01446e186584` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:55.510313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841cdcc7-7efd-4835-9b03-1a517c577b09 None admin] RBAC: Authorizing `identity:get_user(user_id=e94fb5eec4f149e686cb01446e186584)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:55.512591 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841cdcc7-7efd-4835-9b03-1a517c577b09 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:55.515354 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 58/123] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:55 2022] GET /identity/v3/users/e94fb5eec4f149e686cb01446e186584 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:55.516335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cdcc7-7efd-4835-9b03-1a517c577b09 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:55.553868 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:55.573291 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 66/124] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:55.574598 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb465f-0ac3-428c-9583-656345f444f3 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:55.661676 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cdcc7-7efd-4835-9b03-1a517c577b09 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:55.684670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb465f-0ac3-428c-9583-656345f444f3 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:55.713860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee6e1f-ba43-459c-ac90-fa0f10d14a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:55.715042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6e1f-ba43-459c-ac90-fa0f10d14a09 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:55.715485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6e1f-ba43-459c-ac90-fa0f10d14a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:55.716123 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6e1f-ba43-459c-ac90-fa0f10d14a09 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:55.717420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee6e1f-ba43-459c-ac90-fa0f10d14a09 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:55.718169 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee6e1f-ba43-459c-ac90-fa0f10d14a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:55.720514 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee6e1f-ba43-459c-ac90-fa0f10d14a09 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:55.725350 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69965aec-4219-4abc-af0d-8adb66ed24bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:55.726195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965aec-4219-4abc-af0d-8adb66ed24bf None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:55.726606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965aec-4219-4abc-af0d-8adb66ed24bf None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:55.727058 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965aec-4219-4abc-af0d-8adb66ed24bf None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:55.728334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69965aec-4219-4abc-af0d-8adb66ed24bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:55.729734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69965aec-4219-4abc-af0d-8adb66ed24bf None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:55.731975 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 59/125] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:58:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:55.732942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee6e1f-ba43-459c-ac90-fa0f10d14a09 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:55.746138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:55.775167 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 67/126] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 09:58:55 2022] POST /identity/v3/projects => generated 302 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:58:55.777747 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69965aec-4219-4abc-af0d-8adb66ed24bf None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:55.897238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:55.937120 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69965aec-4219-4abc-af0d-8adb66ed24bf None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:55.956413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee6e1f-ba43-459c-ac90-fa0f10d14a09 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:55.982097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e529f-edc8-4b15-a6ac-589189eff8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:55.983812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e529f-edc8-4b15-a6ac-589189eff8ab None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:55.984100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e529f-edc8-4b15-a6ac-589189eff8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:55.984377 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e529f-edc8-4b15-a6ac-589189eff8ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:56.010154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e529f-edc8-4b15-a6ac-589189eff8ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-758e529f-edc8-4b15-a6ac-589189eff8ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:56.023084 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:56.024136 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:58:56.025095 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:58:56.027094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52c3a2-2a08-45fb-911b-ca18ba7957c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:56.027967 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 68/127] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:58:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:56.028201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52c3a2-2a08-45fb-911b-ca18ba7957c0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:56.028620 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52c3a2-2a08-45fb-911b-ca18ba7957c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:56.028987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52c3a2-2a08-45fb-911b-ca18ba7957c0 None admin] PATH_INFO: `/v3/projects/5d649fc3e290424ca5d67aa5529a2393` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:56.029556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e529f-edc8-4b15-a6ac-589189eff8ab None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:56.031245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52c3a2-2a08-45fb-911b-ca18ba7957c0 None admin] RBAC: Authorizing `identity:get_project(project_id=5d649fc3e290424ca5d67aa5529a2393)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:56.033671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52c3a2-2a08-45fb-911b-ca18ba7957c0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:56.035576 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 60/128] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:58:55 2022] GET /identity/v3/projects/5d649fc3e290424ca5d67aa5529a2393 => generated 312 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:56.036462 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52c3a2-2a08-45fb-911b-ca18ba7957c0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:56.102261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:56.158790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e529f-edc8-4b15-a6ac-589189eff8ab None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:56.162583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52c3a2-2a08-45fb-911b-ca18ba7957c0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:56.197183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142ece5-3118-4953-9b84-824fbb95d788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:56.198635 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142ece5-3118-4953-9b84-824fbb95d788 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:56.199207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142ece5-3118-4953-9b84-824fbb95d788 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:56.199676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142ece5-3118-4953-9b84-824fbb95d788 None admin] PATH_INFO: `/v3/users/157cdb8bd93f4162b380bbe5ab80c179` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:56.206071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ae2cf-7c5a-439b-a55f-29396b0640ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:56.206796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae2cf-7c5a-439b-a55f-29396b0640ea None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:56.207087 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae2cf-7c5a-439b-a55f-29396b0640ea None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:56.207677 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae2cf-7c5a-439b-a55f-29396b0640ea None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:56.208452 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ae2cf-7c5a-439b-a55f-29396b0640ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:56.208831 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ae2cf-7c5a-439b-a55f-29396b0640ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:56.210305 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ae2cf-7c5a-439b-a55f-29396b0640ea None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:56.221554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142ece5-3118-4953-9b84-824fbb95d788 None admin] RBAC: Authorizing `identity:get_user(user_id=157cdb8bd93f4162b380bbe5ab80c179)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:56.222424 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 61/129] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:58:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:56.223078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ae2cf-7c5a-439b-a55f-29396b0640ea None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:56.224818 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142ece5-3118-4953-9b84-824fbb95d788 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:56.227067 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 69/130] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:56 2022] GET /identity/v3/users/157cdb8bd93f4162b380bbe5ab80c179 => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:56.228168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142ece5-3118-4953-9b84-824fbb95d788 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:56.242451 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:56.338555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142ece5-3118-4953-9b84-824fbb95d788 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:56.352486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:56.357006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ae2cf-7c5a-439b-a55f-29396b0640ea None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:56.375840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af65c8-7108-4708-aa46-1e11e2e5835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:56.377065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af65c8-7108-4708-aa46-1e11e2e5835c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:56.377535 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af65c8-7108-4708-aa46-1e11e2e5835c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:56.377996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af65c8-7108-4708-aa46-1e11e2e5835c None admin] PATH_INFO: `/v3/users/e94fb5eec4f149e686cb01446e186584` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:56.379425 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af65c8-7108-4708-aa46-1e11e2e5835c None admin] RBAC: Authorizing `identity:get_user(user_id=e94fb5eec4f149e686cb01446e186584)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:56.381192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af65c8-7108-4708-aa46-1e11e2e5835c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:56.383145 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 70/131] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:56 2022] GET /identity/v3/users/e94fb5eec4f149e686cb01446e186584 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:56.384097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af65c8-7108-4708-aa46-1e11e2e5835c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:56.399417 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e698f-7b0f-410e-b6a7-ef625c210a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:56.400783 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e698f-7b0f-410e-b6a7-ef625c210a43 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:56.401083 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e698f-7b0f-410e-b6a7-ef625c210a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:56.401594 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e698f-7b0f-410e-b6a7-ef625c210a43 None admin] PATH_INFO: `/v3/projects/5d649fc3e290424ca5d67aa5529a2393/users/e94fb5eec4f149e686cb01446e186584/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:56.404272 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e698f-7b0f-410e-b6a7-ef625c210a43 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d649fc3e290424ca5d67aa5529a2393, user_id=e94fb5eec4f149e686cb01446e186584, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:56.405784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e698f-7b0f-410e-b6a7-ef625c210a43 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:56.422832 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 62/132] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:58:56 2022] PUT /identity/v3/projects/5d649fc3e290424ca5d67aa5529a2393/users/e94fb5eec4f149e686cb01446e186584/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:58:56.425116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e698f-7b0f-410e-b6a7-ef625c210a43 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:56.608720 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af65c8-7108-4708-aa46-1e11e2e5835c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:56.621794 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:56.695542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe81df-d507-4f06-bfe9-7e0e3aedf50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:56.696481 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe81df-d507-4f06-bfe9-7e0e3aedf50d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:56.696615 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe81df-d507-4f06-bfe9-7e0e3aedf50d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:56.696819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe81df-d507-4f06-bfe9-7e0e3aedf50d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:56.723801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e698f-7b0f-410e-b6a7-ef625c210a43 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:56.764448 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:56.790259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe81df-d507-4f06-bfe9-7e0e3aedf50d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:56.790259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe81df-d507-4f06-bfe9-7e0e3aedf50d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:56.872270 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75247a0e-73f9-4eaa-aed9-ff8bbf66c69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:56.873079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75247a0e-73f9-4eaa-aed9-ff8bbf66c69b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:56.873361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75247a0e-73f9-4eaa-aed9-ff8bbf66c69b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:56.873639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75247a0e-73f9-4eaa-aed9-ff8bbf66c69b None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:56.874931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75247a0e-73f9-4eaa-aed9-ff8bbf66c69b None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:56.876425 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75247a0e-73f9-4eaa-aed9-ff8bbf66c69b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:56.878150 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 63/133] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:58:56 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:56.888159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75247a0e-73f9-4eaa-aed9-ff8bbf66c69b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:56.895572 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 71/134] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:58:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:58:56.903482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe81df-d507-4f06-bfe9-7e0e3aedf50d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:57.012267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:57.031459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe81df-d507-4f06-bfe9-7e0e3aedf50d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:57.068911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ba45b-dfa8-416f-ad81-b6d9f3dd3f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:57.069804 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ba45b-dfa8-416f-ad81-b6d9f3dd3f22 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:57.070090 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ba45b-dfa8-416f-ad81-b6d9f3dd3f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:57.070974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ba45b-dfa8-416f-ad81-b6d9f3dd3f22 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:57.071638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ba45b-dfa8-416f-ad81-b6d9f3dd3f22 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:57.073113 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ba45b-dfa8-416f-ad81-b6d9f3dd3f22 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:57.075175 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 72/135] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:58:56 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:57.080994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ba45b-dfa8-416f-ad81-b6d9f3dd3f22 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:57.185854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ba45b-dfa8-416f-ad81-b6d9f3dd3f22 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:57.220363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3fdc0-e4a5-4c5a-b703-23ce5026779f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:57.221038 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3fdc0-e4a5-4c5a-b703-23ce5026779f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:57.221324 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3fdc0-e4a5-4c5a-b703-23ce5026779f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:57.221601 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3fdc0-e4a5-4c5a-b703-23ce5026779f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:57.222513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3fdc0-e4a5-4c5a-b703-23ce5026779f None admin] RBAC: Adding query filter params (role.id=8646f386c6704cb190f12a4b4e1a2245, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4, user.id=157cdb8bd93f4162b380bbe5ab80c179) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:57.222833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3fdc0-e4a5-4c5a-b703-23ce5026779f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:57.224868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3fdc0-e4a5-4c5a-b703-23ce5026779f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:57.231740 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 73/136] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:58:57 2022] GET /identity/v3/role_assignments?user.id=157cdb8bd93f4162b380bbe5ab80c179&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=8646f386c6704cb190f12a4b4e1a2245 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:57.234398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad212bf-2a91-49c0-8d6e-00e06838c730 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:57.234398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad212bf-2a91-49c0-8d6e-00e06838c730 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:57.234398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad212bf-2a91-49c0-8d6e-00e06838c730 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:57.245432 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 74/137] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:57.252094 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56451ba-8ffc-495f-87c9-6c5513c64bca None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:57.252666 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56451ba-8ffc-495f-87c9-6c5513c64bca None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:57.253177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56451ba-8ffc-495f-87c9-6c5513c64bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:57.264844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75247a0e-73f9-4eaa-aed9-ff8bbf66c69b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:57.295822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-f56451ba-8ffc-495f-87c9-6c5513c64bca None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:57.297104 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f56451ba-8ffc-495f-87c9-6c5513c64bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:57.314424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:57.396513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3addfb-f332-4332-ba8b-cdfb8ae7983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:57.397434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3addfb-f332-4332-ba8b-cdfb8ae7983e None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:57.397717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3addfb-f332-4332-ba8b-cdfb8ae7983e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:57.397994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3addfb-f332-4332-ba8b-cdfb8ae7983e None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/users/e94fb5eec4f149e686cb01446e186584/roles/70d38d0a968a4e9585c5873d24cfbbb1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:57.401092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3addfb-f332-4332-ba8b-cdfb8ae7983e None admin] RBAC: Authorizing `identity:create_grant(project_id=1d72a2d59fc146d3afa2a17c4f4460f4, user_id=e94fb5eec4f149e686cb01446e186584, role_id=70d38d0a968a4e9585c5873d24cfbbb1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:57.403019 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3addfb-f332-4332-ba8b-cdfb8ae7983e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:57.417488 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 75/138] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:58:57.417896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856d80-26df-4dea-a476-e6ac71155603 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:57.418252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856d80-26df-4dea-a476-e6ac71155603 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:57.418566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856d80-26df-4dea-a476-e6ac71155603 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:57.420580 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 76/139] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:57.426426 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294cade-28b0-4474-84cb-1271c27d66ef None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:57.426751 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294cade-28b0-4474-84cb-1271c27d66ef None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:57.427036 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294cade-28b0-4474-84cb-1271c27d66ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:57.429432 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 64/140] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:58:56 2022] PUT /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/users/e94fb5eec4f149e686cb01446e186584/roles/70d38d0a968a4e9585c5873d24cfbbb1 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 09:58:57.458683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a294cade-28b0-4474-84cb-1271c27d66ef None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:57.459947 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a294cade-28b0-4474-84cb-1271c27d66ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:57.462913 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3addfb-f332-4332-ba8b-cdfb8ae7983e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:57.466560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:57.570818 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 77/141] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:58:57.616052 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294cade-28b0-4474-84cb-1271c27d66ef None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:57.666148 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3addfb-f332-4332-ba8b-cdfb8ae7983e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:57.726822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:57.767680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98c252-29b2-4c62-bf01-109c902d4b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:57.767975 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98c252-29b2-4c62-bf01-109c902d4b5c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:57.768259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98c252-29b2-4c62-bf01-109c902d4b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:57.768534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98c252-29b2-4c62-bf01-109c902d4b5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:57.782322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294cade-28b0-4474-84cb-1271c27d66ef None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:57.814118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98c252-29b2-4c62-bf01-109c902d4b5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-0e98c252-29b2-4c62-bf01-109c902d4b5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:57.822017 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:57.822925 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:58:57.823690 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:58:57.831439 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 65/142] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:58:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:57.838694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98c252-29b2-4c62-bf01-109c902d4b5c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:57.840164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df14396-c8a5-4edf-95c4-cf9e9689ba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:57.840993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df14396-c8a5-4edf-95c4-cf9e9689ba4c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:57.841279 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df14396-c8a5-4edf-95c4-cf9e9689ba4c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:57.841552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df14396-c8a5-4edf-95c4-cf9e9689ba4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:57.854235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df14396-c8a5-4edf-95c4-cf9e9689ba4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-7df14396-c8a5-4edf-95c4-cf9e9689ba4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:58:57.869993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:58:57.870967 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:58:57.872291 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 78/143] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:58:57.877724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df14396-c8a5-4edf-95c4-cf9e9689ba4c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:58.019896 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:58.055786 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df14396-c8a5-4edf-95c4-cf9e9689ba4c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:58.110592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:58.116868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd5b58-f84e-4027-a865-d68d46a6cc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:58.117611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5b58-f84e-4027-a865-d68d46a6cc09 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:58.117902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5b58-f84e-4027-a865-d68d46a6cc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:58.118177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5b58-f84e-4027-a865-d68d46a6cc09 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:58.119110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd5b58-f84e-4027-a865-d68d46a6cc09 None admin] RBAC: Adding query filter params (name=member) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:58.119547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd5b58-f84e-4027-a865-d68d46a6cc09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:58.121191 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd5b58-f84e-4027-a865-d68d46a6cc09 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:58.137562 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 79/144] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:58:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:58.143248 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd5b58-f84e-4027-a865-d68d46a6cc09 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:58.144171 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98c252-29b2-4c62-bf01-109c902d4b5c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:58.235781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e5ebe-c00c-492e-bd4e-f67319d28113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:58.236467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5ebe-c00c-492e-bd4e-f67319d28113 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:58.236751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5ebe-c00c-492e-bd4e-f67319d28113 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:58.237029 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5ebe-c00c-492e-bd4e-f67319d28113 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:58.237791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e5ebe-c00c-492e-bd4e-f67319d28113 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:58.238099 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e5ebe-c00c-492e-bd4e-f67319d28113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:58.239527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e5ebe-c00c-492e-bd4e-f67319d28113 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:58.260662 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 66/145] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:58:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:58.266606 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e5ebe-c00c-492e-bd4e-f67319d28113 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:58.350032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd5b58-f84e-4027-a865-d68d46a6cc09 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:58.435447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b650d-d19a-458e-8236-ce9a3e27fc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:58.436345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b650d-d19a-458e-8236-ce9a3e27fc41 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:58.436622 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b650d-d19a-458e-8236-ce9a3e27fc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:58.436892 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b650d-d19a-458e-8236-ce9a3e27fc41 None admin] PATH_INFO: `/v3/users/e94fb5eec4f149e686cb01446e186584` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:58.438102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b650d-d19a-458e-8236-ce9a3e27fc41 None admin] RBAC: Authorizing `identity:get_user(user_id=e94fb5eec4f149e686cb01446e186584)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:58.439581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b650d-d19a-458e-8236-ce9a3e27fc41 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:58.441531 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 80/146] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:58 2022] GET /identity/v3/users/e94fb5eec4f149e686cb01446e186584 => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:58:58.448356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b650d-d19a-458e-8236-ce9a3e27fc41 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:58.451410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e5ebe-c00c-492e-bd4e-f67319d28113 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:58.495231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:58.515456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dab3c-9a0e-478e-9169-e110c15e0df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:58.516406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dab3c-9a0e-478e-9169-e110c15e0df4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:58.516657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dab3c-9a0e-478e-9169-e110c15e0df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:58.516934 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dab3c-9a0e-478e-9169-e110c15e0df4 None admin] PATH_INFO: `/v3/users/c042e1f397444e8991d3f0b7bfb28908` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:58.518140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30dab3c-9a0e-478e-9169-e110c15e0df4 None admin] RBAC: Authorizing `identity:get_user(user_id=c042e1f397444e8991d3f0b7bfb28908)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:58.519638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30dab3c-9a0e-478e-9169-e110c15e0df4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:58.521641 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 67/147] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:58:58 2022] GET /identity/v3/users/c042e1f397444e8991d3f0b7bfb28908 => generated 247 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:58.535899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dab3c-9a0e-478e-9169-e110c15e0df4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:58.585833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:58.681942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b650d-d19a-458e-8236-ce9a3e27fc41 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:58.713225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dab3c-9a0e-478e-9169-e110c15e0df4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:58.782882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deca021-0794-46b2-937d-37e334e6b968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:58.783565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deca021-0794-46b2-937d-37e334e6b968 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:58.783845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deca021-0794-46b2-937d-37e334e6b968 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:58.784222 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deca021-0794-46b2-937d-37e334e6b968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:58.785365 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd546ed5-e36f-4e11-913d-6dac793a7b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:58.786029 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546ed5-e36f-4e11-913d-6dac793a7b36 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:58.786339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546ed5-e36f-4e11-913d-6dac793a7b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:58.786626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546ed5-e36f-4e11-913d-6dac793a7b36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:58.925877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deca021-0794-46b2-937d-37e334e6b968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:58.927898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deca021-0794-46b2-937d-37e334e6b968 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:58.943490 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:58.972256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd546ed5-e36f-4e11-913d-6dac793a7b36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:58.974712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd546ed5-e36f-4e11-913d-6dac793a7b36 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:59.062581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:59.133025 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 81/148] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 09:58:59.139189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deca021-0794-46b2-937d-37e334e6b968 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:59.184475 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 68/149] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 09:58:59.186074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db3e2-937a-4fce-b414-b5da800e1868 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:59.186618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db3e2-937a-4fce-b414-b5da800e1868 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:59.187115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db3e2-937a-4fce-b414-b5da800e1868 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:59.188739 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 69/150] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:59.193738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38db3e2-937a-4fce-b414-b5da800e1868 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:59.345013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deca021-0794-46b2-937d-37e334e6b968 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:59.348163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:59.364815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38db3e2-937a-4fce-b414-b5da800e1868 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:59.409456 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8315d-089b-4c15-9bcb-a0d3f0e4f3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:59.411724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8315d-089b-4c15-9bcb-a0d3f0e4f3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:59.412472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8315d-089b-4c15-9bcb-a0d3f0e4f3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:59.413698 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8315d-089b-4c15-9bcb-a0d3f0e4f3c3 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:59.415815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8315d-089b-4c15-9bcb-a0d3f0e4f3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:59.418477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8315d-089b-4c15-9bcb-a0d3f0e4f3c3 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:59.420601 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 82/151] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:58:59 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:59.427647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b9379-693f-4966-8653-1f22c842a723 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:59.428320 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b9379-693f-4966-8653-1f22c842a723 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:59.429940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b9379-693f-4966-8653-1f22c842a723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:59.438454 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32f605-430d-4cf0-bbe1-62ef9664456c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:59.439644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32f605-430d-4cf0-bbe1-62ef9664456c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:59.440393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32f605-430d-4cf0-bbe1-62ef9664456c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:59.440865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32f605-430d-4cf0-bbe1-62ef9664456c None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:59.442330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32f605-430d-4cf0-bbe1-62ef9664456c None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:59.445223 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32f605-430d-4cf0-bbe1-62ef9664456c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:59.447325 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 70/152] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:58:59 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:59.448307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32f605-430d-4cf0-bbe1-62ef9664456c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:59.467252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-111b9379-693f-4966-8653-1f22c842a723 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:59.468482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-111b9379-693f-4966-8653-1f22c842a723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:59.509966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:59.578658 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 83/153] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:59.583555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b9379-693f-4966-8653-1f22c842a723 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:59.608471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32f605-430d-4cf0-bbe1-62ef9664456c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:59.645530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68b00f-4a7d-49f1-8d71-29867266947f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:59.646558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68b00f-4a7d-49f1-8d71-29867266947f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:59.647291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68b00f-4a7d-49f1-8d71-29867266947f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:59.647418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68b00f-4a7d-49f1-8d71-29867266947f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:59.648366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68b00f-4a7d-49f1-8d71-29867266947f None admin] RBAC: Adding query filter params (role.id=b10cd0627cc84a9c92dba333901e60e1, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4, user.id=e94fb5eec4f149e686cb01446e186584) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:59.648783 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68b00f-4a7d-49f1-8d71-29867266947f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:59.650477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68b00f-4a7d-49f1-8d71-29867266947f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:59.657473 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 71/154] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:58:59 2022] GET /identity/v3/role_assignments?user.id=e94fb5eec4f149e686cb01446e186584&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:59.664578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cc6fd-639a-413b-97ff-9cd023938308 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:59.664578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cc6fd-639a-413b-97ff-9cd023938308 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:59.664578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cc6fd-639a-413b-97ff-9cd023938308 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:59.664578 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 72/155] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:59.664578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cc6fd-639a-413b-97ff-9cd023938308 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:59.727221 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:59.751914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b9379-693f-4966-8653-1f22c842a723 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:59.831849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f3c4e-b30a-46f7-b4ec-82dfaff6eaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:59.832896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f3c4e-b30a-46f7-b4ec-82dfaff6eaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:59.833181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f3c4e-b30a-46f7-b4ec-82dfaff6eaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:59.833457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f3c4e-b30a-46f7-b4ec-82dfaff6eaf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:59.834548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f3c4e-b30a-46f7-b4ec-82dfaff6eaf0 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4, user.id=c042e1f397444e8991d3f0b7bfb28908) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:58:59.834815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f3c4e-b30a-46f7-b4ec-82dfaff6eaf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:59.836292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f3c4e-b30a-46f7-b4ec-82dfaff6eaf0 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:59.842937 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 84/156] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:58:59 2022] GET /identity/v3/role_assignments?user.id=c042e1f397444e8991d3f0b7bfb28908&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=c184a177010144429f495f3290d28221 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:59.844324 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a7274-a94b-473a-90cc-7c62fccee519 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:59.844616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a7274-a94b-473a-90cc-7c62fccee519 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:59.844890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a7274-a94b-473a-90cc-7c62fccee519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:59.861730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cc6fd-639a-413b-97ff-9cd023938308 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:58:59.862729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e11a7274-a94b-473a-90cc-7c62fccee519 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:59.863966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e11a7274-a94b-473a-90cc-7c62fccee519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:58:59.884513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:58:59.905412 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8965200-0398-41f9-9b69-16e737bd8fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:58:59.906255 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8965200-0398-41f9-9b69-16e737bd8fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:59.906538 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8965200-0398-41f9-9b69-16e737bd8fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:59.906866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8965200-0398-41f9-9b69-16e737bd8fb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:59.908175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8965200-0398-41f9-9b69-16e737bd8fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:58:59.909989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8965200-0398-41f9-9b69-16e737bd8fb5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:58:59.912211 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 73/157] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:58:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:58:59.918766 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4364a25-a873-4627-b151-8ec68441c7ab None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:59.919404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4364a25-a873-4627-b151-8ec68441c7ab None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:59.919807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4364a25-a873-4627-b151-8ec68441c7ab None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:59.921933 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 74/158] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:58:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:58:59.924417 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4364a25-a873-4627-b151-8ec68441c7ab None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:58:59.945837 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 85/159] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:58:59.947061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f136e-26ea-459e-8924-63833f2ef033 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:58:59.947353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f136e-26ea-459e-8924-63833f2ef033 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:58:59.947630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f136e-26ea-459e-8924-63833f2ef033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:58:59.967854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a16f136e-26ea-459e-8924-63833f2ef033 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:58:59.969143 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a16f136e-26ea-459e-8924-63833f2ef033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:00.062752 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 86/160] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:00.064399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f136e-26ea-459e-8924-63833f2ef033 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:00.069779 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:00.084443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4364a25-a873-4627-b151-8ec68441c7ab None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:00.131206 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e9db9-e90b-4885-82f0-a89d74175c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:00.132313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e9db9-e90b-4885-82f0-a89d74175c9d None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:00.132817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e9db9-e90b-4885-82f0-a89d74175c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:00.133284 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e9db9-e90b-4885-82f0-a89d74175c9d None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:00.134213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e9db9-e90b-4885-82f0-a89d74175c9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:00.135931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e9db9-e90b-4885-82f0-a89d74175c9d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:00.164964 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 75/161] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:58:59 2022] POST /identity/v3/users => generated 283 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 09:59:00.165712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e9db9-e90b-4885-82f0-a89d74175c9d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:00.181636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:00.206127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f136e-26ea-459e-8924-63833f2ef033 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:00.244247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d69205-107d-42e1-84cb-3c5d818bdf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:00.245085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d69205-107d-42e1-84cb-3c5d818bdf3c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:00.245372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d69205-107d-42e1-84cb-3c5d818bdf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:00.245646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d69205-107d-42e1-84cb-3c5d818bdf3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:00.257238 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d69205-107d-42e1-84cb-3c5d818bdf3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-c5d69205-107d-42e1-84cb-3c5d818bdf3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:00.265200 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:00.266157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:00.266995 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 87/162] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:00.267812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d177f-53ed-4ad6-af48-db1fb5f403f6 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:00.268421 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d177f-53ed-4ad6-af48-db1fb5f403f6 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:00.268421 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d177f-53ed-4ad6-af48-db1fb5f403f6 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:00.269696 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 88/163] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:00.274028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d177f-53ed-4ad6-af48-db1fb5f403f6 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:00.294827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e9db9-e90b-4885-82f0-a89d74175c9d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:00.335005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae3c78-3406-460f-876f-aaed9345c59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:00.336072 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae3c78-3406-460f-876f-aaed9345c59e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:00.336358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae3c78-3406-460f-876f-aaed9345c59e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:00.336635 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae3c78-3406-460f-876f-aaed9345c59e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:00.353272 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae3c78-3406-460f-876f-aaed9345c59e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6dae3c78-3406-460f-876f-aaed9345c59e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:00.361272 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:00.362310 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:00.363188 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 76/164] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:00.363969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab25c25-6fd5-4ea4-a7cf-1885274489dd None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:00.364054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab25c25-6fd5-4ea4-a7cf-1885274489dd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:00.364299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab25c25-6fd5-4ea4-a7cf-1885274489dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:00.383571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fab25c25-6fd5-4ea4-a7cf-1885274489dd None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:00.385075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fab25c25-6fd5-4ea4-a7cf-1885274489dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:00.387197 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:00.396780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d177f-53ed-4ad6-af48-db1fb5f403f6 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:00.439353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60644667-fcbb-4b4f-8811-2a6819c88dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:00.440178 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60644667-fcbb-4b4f-8811-2a6819c88dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:00.440542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60644667-fcbb-4b4f-8811-2a6819c88dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:00.440825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60644667-fcbb-4b4f-8811-2a6819c88dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:00.441592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60644667-fcbb-4b4f-8811-2a6819c88dd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:00.441899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60644667-fcbb-4b4f-8811-2a6819c88dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:00.443523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60644667-fcbb-4b4f-8811-2a6819c88dd0 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:00.454076 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 89/165] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:00.455100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60644667-fcbb-4b4f-8811-2a6819c88dd0 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:00.466684 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:00.475667 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 77/166] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:00.476782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab25c25-6fd5-4ea4-a7cf-1885274489dd None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:00.575741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60644667-fcbb-4b4f-8811-2a6819c88dd0 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:00.612389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab25c25-6fd5-4ea4-a7cf-1885274489dd None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:00.622717 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d6c80-8559-4689-8b09-a9f4546e222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:00.623656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d6c80-8559-4689-8b09-a9f4546e222d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:00.624107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d6c80-8559-4689-8b09-a9f4546e222d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:00.624898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d6c80-8559-4689-8b09-a9f4546e222d None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:00.626131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d6c80-8559-4689-8b09-a9f4546e222d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:00.626624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d6c80-8559-4689-8b09-a9f4546e222d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:00.628504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d6c80-8559-4689-8b09-a9f4546e222d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:00.639289 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 90/167] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:00.644518 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d6c80-8559-4689-8b09-a9f4546e222d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:00.651541 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b999a-bc27-4688-a030-198e5c0cf120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:00.652441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b999a-bc27-4688-a030-198e5c0cf120 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:00.652733 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b999a-bc27-4688-a030-198e5c0cf120 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:00.653007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b999a-bc27-4688-a030-198e5c0cf120 None admin] PATH_INFO: `/v3/users/e94fb5eec4f149e686cb01446e186584` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:00.654354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b999a-bc27-4688-a030-198e5c0cf120 None admin] RBAC: Authorizing `identity:get_user(user_id=e94fb5eec4f149e686cb01446e186584)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:00.655786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b999a-bc27-4688-a030-198e5c0cf120 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:00.657458 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 78/168] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:00 2022] GET /identity/v3/users/e94fb5eec4f149e686cb01446e186584 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:00.658249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b999a-bc27-4688-a030-198e5c0cf120 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:00.671556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:00.733126 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:00.786428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d6c80-8559-4689-8b09-a9f4546e222d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:00.794449 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b999a-bc27-4688-a030-198e5c0cf120 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:00.829189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b61b45-b8f1-4be4-9494-f212b2e4e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:00.830201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b61b45-b8f1-4be4-9494-f212b2e4e748 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:00.830594 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b61b45-b8f1-4be4-9494-f212b2e4e748 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:00.830872 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b61b45-b8f1-4be4-9494-f212b2e4e748 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:00.831035 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3584830-d0f4-40a0-8127-fa9f88f8e915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:00.831891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3584830-d0f4-40a0-8127-fa9f88f8e915 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:00.832175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3584830-d0f4-40a0-8127-fa9f88f8e915 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:00.832796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3584830-d0f4-40a0-8127-fa9f88f8e915 None admin] PATH_INFO: `/v3/users/157cdb8bd93f4162b380bbe5ab80c179` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:00.834064 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3584830-d0f4-40a0-8127-fa9f88f8e915 None admin] RBAC: Authorizing `identity:get_user(user_id=157cdb8bd93f4162b380bbe5ab80c179)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:00.835578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3584830-d0f4-40a0-8127-fa9f88f8e915 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:00.837241 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 79/169] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:00 2022] GET /identity/v3/users/157cdb8bd93f4162b380bbe5ab80c179 => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:00.838024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3584830-d0f4-40a0-8127-fa9f88f8e915 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:00.844266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b61b45-b8f1-4be4-9494-f212b2e4e748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-69b61b45-b8f1-4be4-9494-f212b2e4e748 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:00.855187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:00.856105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:00.857338 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 91/170] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:59:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:00.858343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b61b45-b8f1-4be4-9494-f212b2e4e748 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:00.918080 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:00.945117 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3584830-d0f4-40a0-8127-fa9f88f8e915 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:00.981370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60b746-3856-4e91-87a8-f246fd36606b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:00.982039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60b746-3856-4e91-87a8-f246fd36606b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:00.982338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60b746-3856-4e91-87a8-f246fd36606b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:00.982624 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60b746-3856-4e91-87a8-f246fd36606b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:00.994147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b61b45-b8f1-4be4-9494-f212b2e4e748 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:01.021217 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:01.030990 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4d1e0-28ff-4c9f-8b1b-2787cd45c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:01.032218 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d1e0-28ff-4c9f-8b1b-2787cd45c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:01.032691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d1e0-28ff-4c9f-8b1b-2787cd45c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:01.033153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d1e0-28ff-4c9f-8b1b-2787cd45c4f2 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:01.034585 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4d1e0-28ff-4c9f-8b1b-2787cd45c4f2 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:01.036234 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4d1e0-28ff-4c9f-8b1b-2787cd45c4f2 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:01.038294 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 92/171] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:00 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:01.039256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4d1e0-28ff-4c9f-8b1b-2787cd45c4f2 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:01.063790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60b746-3856-4e91-87a8-f246fd36606b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:01.065596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60b746-3856-4e91-87a8-f246fd36606b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:01.142718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4d1e0-28ff-4c9f-8b1b-2787cd45c4f2 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:01.150952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:01.178753 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 80/172] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:01.179510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60b746-3856-4e91-87a8-f246fd36606b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:01.185800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb3da3-9aa9-4bfe-bdd5-0086d2875c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:01.186840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb3da3-9aa9-4bfe-bdd5-0086d2875c3d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:01.187349 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb3da3-9aa9-4bfe-bdd5-0086d2875c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:01.187813 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb3da3-9aa9-4bfe-bdd5-0086d2875c3d None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:01.188762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb3da3-9aa9-4bfe-bdd5-0086d2875c3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:01.189328 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb3da3-9aa9-4bfe-bdd5-0086d2875c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:01.190946 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb3da3-9aa9-4bfe-bdd5-0086d2875c3d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:01.199954 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 93/173] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:01.201110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb3da3-9aa9-4bfe-bdd5-0086d2875c3d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:01.260705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:01.284197 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60b746-3856-4e91-87a8-f246fd36606b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:01.305570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb3da3-9aa9-4bfe-bdd5-0086d2875c3d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:01.319618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8a752-4637-4d68-bb48-4fea2e368343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:01.320535 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8a752-4637-4d68-bb48-4fea2e368343 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:01.320836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8a752-4637-4d68-bb48-4fea2e368343 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:01.321299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8a752-4637-4d68-bb48-4fea2e368343 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/users/157cdb8bd93f4162b380bbe5ab80c179/roles/8646f386c6704cb190f12a4b4e1a2245` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:01.323129 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8a752-4637-4d68-bb48-4fea2e368343 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d72a2d59fc146d3afa2a17c4f4460f4, user_id=157cdb8bd93f4162b380bbe5ab80c179, role_id=8646f386c6704cb190f12a4b4e1a2245)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:01.324612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8a752-4637-4d68-bb48-4fea2e368343 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:01.346304 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 81/174] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:59:01 2022] PUT /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/users/157cdb8bd93f4162b380bbe5ab80c179/roles/8646f386c6704cb190f12a4b4e1a2245 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:01.347208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8a752-4637-4d68-bb48-4fea2e368343 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:01.357106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f504c6-f605-4a73-9683-9c1e48210586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:01.359249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f504c6-f605-4a73-9683-9c1e48210586 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:01.359541 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f504c6-f605-4a73-9683-9c1e48210586 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:01.359819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f504c6-f605-4a73-9683-9c1e48210586 None admin] PATH_INFO: `/v3/projects/5d649fc3e290424ca5d67aa5529a2393` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:01.364246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f504c6-f605-4a73-9683-9c1e48210586 None admin] RBAC: Authorizing `identity:get_project(project_id=5d649fc3e290424ca5d67aa5529a2393)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:01.367041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f504c6-f605-4a73-9683-9c1e48210586 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:01.373357 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 94/175] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:01 2022] GET /identity/v3/projects/5d649fc3e290424ca5d67aa5529a2393 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:01.383958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f504c6-f605-4a73-9683-9c1e48210586 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:01.444962 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:01.508147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f504c6-f605-4a73-9683-9c1e48210586 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:01.539663 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:01.545062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc68f53-be96-4fa5-8725-9ef0c99d20ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:01.545757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc68f53-be96-4fa5-8725-9ef0c99d20ee None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:01.546048 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc68f53-be96-4fa5-8725-9ef0c99d20ee None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:01.546359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc68f53-be96-4fa5-8725-9ef0c99d20ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:01.547208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc68f53-be96-4fa5-8725-9ef0c99d20ee None admin] RBAC: Adding query filter params (role.id=b10cd0627cc84a9c92dba333901e60e1, scope.project.id=5d649fc3e290424ca5d67aa5529a2393, user.id=e94fb5eec4f149e686cb01446e186584) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:01.547522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc68f53-be96-4fa5-8725-9ef0c99d20ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:01.549007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc68f53-be96-4fa5-8725-9ef0c99d20ee None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:01.555979 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 95/176] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:01 2022] GET /identity/v3/role_assignments?user.id=e94fb5eec4f149e686cb01446e186584&scope.project.id=5d649fc3e290424ca5d67aa5529a2393&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:01.599467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8a752-4637-4d68-bb48-4fea2e368343 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:01.660996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415553c-5841-4608-a93d-20e06a846a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:01.662016 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415553c-5841-4608-a93d-20e06a846a46 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:01.662332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415553c-5841-4608-a93d-20e06a846a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:01.662616 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415553c-5841-4608-a93d-20e06a846a46 None admin] PATH_INFO: `/v3/users/e94fb5eec4f149e686cb01446e186584` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:01.668151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415553c-5841-4608-a93d-20e06a846a46 None admin] RBAC: Authorizing `identity:get_user(user_id=e94fb5eec4f149e686cb01446e186584)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:01.668918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415553c-5841-4608-a93d-20e06a846a46 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:01.670879 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 82/177] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:01 2022] GET /identity/v3/users/e94fb5eec4f149e686cb01446e186584 => generated 275 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:01.676683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc68f53-be96-4fa5-8725-9ef0c99d20ee None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:01.786248 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc68f53-be96-4fa5-8725-9ef0c99d20ee None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:01.821827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbeda81-a60e-4359-911f-39324c0caadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:01.822555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbeda81-a60e-4359-911f-39324c0caadc None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:01.822840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbeda81-a60e-4359-911f-39324c0caadc None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:01.823118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbeda81-a60e-4359-911f-39324c0caadc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:01.884930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:01.897168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbeda81-a60e-4359-911f-39324c0caadc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:01.898767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbeda81-a60e-4359-911f-39324c0caadc None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:02.002279 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 96/178] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:02.008054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415553c-5841-4608-a93d-20e06a846a46 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:02.112076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415553c-5841-4608-a93d-20e06a846a46 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:02.113535 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:02.148909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71323baf-894c-442b-affb-4fa13e5a1866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:02.149799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71323baf-894c-442b-affb-4fa13e5a1866 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:02.150087 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71323baf-894c-442b-affb-4fa13e5a1866 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:02.150491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71323baf-894c-442b-affb-4fa13e5a1866 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:02.151738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71323baf-894c-442b-affb-4fa13e5a1866 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:02.153219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71323baf-894c-442b-affb-4fa13e5a1866 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:02.154991 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 83/179] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:02 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:02.160821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbeda81-a60e-4359-911f-39324c0caadc None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:02.266638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:02.270819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbeda81-a60e-4359-911f-39324c0caadc None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:02.314774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fce03c-8f92-440f-b159-65d2f013c622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:02.315555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fce03c-8f92-440f-b159-65d2f013c622 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:02.315848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fce03c-8f92-440f-b159-65d2f013c622 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:02.316195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fce03c-8f92-440f-b159-65d2f013c622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:02.317181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fce03c-8f92-440f-b159-65d2f013c622 None admin] RBAC: Adding query filter params (role.id=70d38d0a968a4e9585c5873d24cfbbb1, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4, user.id=e94fb5eec4f149e686cb01446e186584) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:02.317522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fce03c-8f92-440f-b159-65d2f013c622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:02.319294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fce03c-8f92-440f-b159-65d2f013c622 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:02.326785 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 97/180] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:02 2022] GET /identity/v3/role_assignments?user.id=e94fb5eec4f149e686cb01446e186584&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=70d38d0a968a4e9585c5873d24cfbbb1 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:02.810000 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7e0a6-a45f-4a33-8244-ef30119d7d74 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:02.810597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7e0a6-a45f-4a33-8244-ef30119d7d74 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:02.810597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7e0a6-a45f-4a33-8244-ef30119d7d74 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:02.812154 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 84/181] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:02.817582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d76980-b3bc-4d8c-bbd1-274e84272092 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:02.817923 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d76980-b3bc-4d8c-bbd1-274e84272092 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:02.818207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d76980-b3bc-4d8c-bbd1-274e84272092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:02.842278 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-19d76980-b3bc-4d8c-bbd1-274e84272092 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:02.843802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-19d76980-b3bc-4d8c-bbd1-274e84272092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:02.917737 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 98/182] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:02.940885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7e0a6-a45f-4a33-8244-ef30119d7d74 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:03.047474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7e0a6-a45f-4a33-8244-ef30119d7d74 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:03.063292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1881-ca16-42ad-a5a2-f5da62c7f2b3 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.063631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1881-ca16-42ad-a5a2-f5da62c7f2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.063851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1881-ca16-42ad-a5a2-f5da62c7f2b3 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.065511 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 99/183] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:03.070706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144c95-ef41-4919-b65c-a70f68acf0d5 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.070945 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144c95-ef41-4919-b65c-a70f68acf0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.071158 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144c95-ef41-4919-b65c-a70f68acf0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.089201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65191d2-f828-402c-b523-c479d083400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:03.089420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a1144c95-ef41-4919-b65c-a70f68acf0d5 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:03.090016 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65191d2-f828-402c-b523-c479d083400c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.090333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65191d2-f828-402c-b523-c479d083400c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.090615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65191d2-f828-402c-b523-c479d083400c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.090723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a1144c95-ef41-4919-b65c-a70f68acf0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:03.104056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65191d2-f828-402c-b523-c479d083400c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c65191d2-f828-402c-b523-c479d083400c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:03.114079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:03.115067 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:03.115853 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:03.116462 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 85/184] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:59:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:03.122725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65191d2-f828-402c-b523-c479d083400c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:03.156539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:03.163464 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 100/185] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:03.187340 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1144c95-ef41-4919-b65c-a70f68acf0d5 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:03.203132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:03.230601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65191d2-f828-402c-b523-c479d083400c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:03.266060 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d5ab4-2704-42d0-9416-20d091c8efcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:03.266803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d5ab4-2704-42d0-9416-20d091c8efcc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.267091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d5ab4-2704-42d0-9416-20d091c8efcc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.267366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d5ab4-2704-42d0-9416-20d091c8efcc None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.268316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d5ab4-2704-42d0-9416-20d091c8efcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:03.268627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d5ab4-2704-42d0-9416-20d091c8efcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:03.270023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d5ab4-2704-42d0-9416-20d091c8efcc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:03.278784 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 86/186] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:03.284475 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d5ab4-2704-42d0-9416-20d091c8efcc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:03.292489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1144c95-ef41-4919-b65c-a70f68acf0d5 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:03.326608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949bf59-aaf2-4725-a605-8a600218d62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:03.327363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949bf59-aaf2-4725-a605-8a600218d62f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.327641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949bf59-aaf2-4725-a605-8a600218d62f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.327912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949bf59-aaf2-4725-a605-8a600218d62f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.340970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949bf59-aaf2-4725-a605-8a600218d62f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-4949bf59-aaf2-4725-a605-8a600218d62f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:03.351339 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:03.353105 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:03.354373 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 101/187] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:59:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:03.359694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949bf59-aaf2-4725-a605-8a600218d62f None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:03.385961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:03.397877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d5ab4-2704-42d0-9416-20d091c8efcc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:03.434454 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad6f85-1180-4346-9e7f-86e054b942cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:03.435362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6f85-1180-4346-9e7f-86e054b942cf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.435654 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6f85-1180-4346-9e7f-86e054b942cf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.435927 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6f85-1180-4346-9e7f-86e054b942cf None admin] PATH_INFO: `/v3/users/588a9fdcdd234ceab4e72c361de4360e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.441216 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:03.449425 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad6f85-1180-4346-9e7f-86e054b942cf None admin] RBAC: Authorizing `identity:get_user(user_id=588a9fdcdd234ceab4e72c361de4360e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:03.450932 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad6f85-1180-4346-9e7f-86e054b942cf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:03.452616 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 87/188] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:03 2022] GET /identity/v3/users/588a9fdcdd234ceab4e72c361de4360e => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:03.458672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b9cf2-e2f7-4dfe-aafb-3f4a333cfba2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.458989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b9cf2-e2f7-4dfe-aafb-3f4a333cfba2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.459274 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b9cf2-e2f7-4dfe-aafb-3f4a333cfba2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.460671 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 88/189] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:03.461313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b9cf2-e2f7-4dfe-aafb-3f4a333cfba2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:03.471763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949bf59-aaf2-4725-a605-8a600218d62f None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:03.508767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabeb9a-0ded-48aa-bf97-525452802c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:03.509764 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabeb9a-0ded-48aa-bf97-525452802c55 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.510060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabeb9a-0ded-48aa-bf97-525452802c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.510567 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabeb9a-0ded-48aa-bf97-525452802c55 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.511284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabeb9a-0ded-48aa-bf97-525452802c55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:03.511596 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabeb9a-0ded-48aa-bf97-525452802c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:03.513035 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabeb9a-0ded-48aa-bf97-525452802c55 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:03.525509 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 102/190] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:03.525509 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b8279-d613-452a-acf2-e4223e3dff38 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.525509 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b8279-d613-452a-acf2-e4223e3dff38 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.525509 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b8279-d613-452a-acf2-e4223e3dff38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.542318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-f70b8279-d613-452a-acf2-e4223e3dff38 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:03.543503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f70b8279-d613-452a-acf2-e4223e3dff38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:03.573964 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b9cf2-e2f7-4dfe-aafb-3f4a333cfba2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:03.610411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6113c0-109a-41fe-a0a6-76b05836bd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:03.611003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6113c0-109a-41fe-a0a6-76b05836bd43 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.611291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6113c0-109a-41fe-a0a6-76b05836bd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.611569 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6113c0-109a-41fe-a0a6-76b05836bd43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.615279 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 103/191] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:03.616255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b8279-d613-452a-acf2-e4223e3dff38 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:03.644137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:03.685180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:03.694733 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6113c0-109a-41fe-a0a6-76b05836bd43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:03.696295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6113c0-109a-41fe-a0a6-76b05836bd43 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:03.723466 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b8279-d613-452a-acf2-e4223e3dff38 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:03.764623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2c5dd-0f22-453a-96ec-0fd441ee27cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:03.765531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2c5dd-0f22-453a-96ec-0fd441ee27cd None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.765814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2c5dd-0f22-453a-96ec-0fd441ee27cd None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.766092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2c5dd-0f22-453a-96ec-0fd441ee27cd None admin] PATH_INFO: `/v3/users/588a9fdcdd234ceab4e72c361de4360e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.767317 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2c5dd-0f22-453a-96ec-0fd441ee27cd None admin] RBAC: Authorizing `identity:get_user(user_id=588a9fdcdd234ceab4e72c361de4360e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:03.768771 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2c5dd-0f22-453a-96ec-0fd441ee27cd None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:03.774699 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 104/192] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:03 2022] GET /identity/v3/users/588a9fdcdd234ceab4e72c361de4360e => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:03.774699 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2c5dd-0f22-453a-96ec-0fd441ee27cd None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:03.809803 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 89/193] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:03.811011 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6113c0-109a-41fe-a0a6-76b05836bd43 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:03.860022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:03.878257 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2c5dd-0f22-453a-96ec-0fd441ee27cd None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:03.913520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6113c0-109a-41fe-a0a6-76b05836bd43 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:03.916051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2bd6a-a2e3-4720-8bc2-c4bdf1c1360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:03.916868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2bd6a-a2e3-4720-8bc2-c4bdf1c1360b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.917152 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2bd6a-a2e3-4720-8bc2-c4bdf1c1360b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.917425 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2bd6a-a2e3-4720-8bc2-c4bdf1c1360b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:03.918615 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2bd6a-a2e3-4720-8bc2-c4bdf1c1360b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:03.920068 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2bd6a-a2e3-4720-8bc2-c4bdf1c1360b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:03.921690 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 105/194] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:03.926826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2bd6a-a2e3-4720-8bc2-c4bdf1c1360b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:03.929543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:03.953294 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24207f70-199e-4c96-a8c0-094b9cb26f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:03.954406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207f70-199e-4c96-a8c0-094b9cb26f87 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:03.954808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207f70-199e-4c96-a8c0-094b9cb26f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:03.955085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207f70-199e-4c96-a8c0-094b9cb26f87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:04.029933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24207f70-199e-4c96-a8c0-094b9cb26f87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:04.031450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24207f70-199e-4c96-a8c0-094b9cb26f87 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:04.033004 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2bd6a-a2e3-4720-8bc2-c4bdf1c1360b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:04.068043 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b4927-23ff-4351-b0e5-2388e2f51a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:04.069002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4927-23ff-4351-b0e5-2388e2f51a5d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:04.069282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4927-23ff-4351-b0e5-2388e2f51a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:04.069554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4927-23ff-4351-b0e5-2388e2f51a5d None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:04.070762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b4927-23ff-4351-b0e5-2388e2f51a5d None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:04.072210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b4927-23ff-4351-b0e5-2388e2f51a5d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:04.073906 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 106/195] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:03 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:04.074661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b4927-23ff-4351-b0e5-2388e2f51a5d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:04.087627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:04.143157 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 90/196] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:04.143157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3230-381a-4703-a49c-88d696c7f6ca None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:04.143157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3230-381a-4703-a49c-88d696c7f6ca None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:04.143157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3230-381a-4703-a49c-88d696c7f6ca None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:04.143157 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 91/197] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:04.143157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a3230-381a-4703-a49c-88d696c7f6ca None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:04.147430 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:04.171674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b4927-23ff-4351-b0e5-2388e2f51a5d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:04.205880 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04ad12-126d-42ff-a3cc-90874fe5e68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:04.206599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04ad12-126d-42ff-a3cc-90874fe5e68c None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:04.206874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04ad12-126d-42ff-a3cc-90874fe5e68c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:04.207163 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04ad12-126d-42ff-a3cc-90874fe5e68c None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:04.208203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04ad12-126d-42ff-a3cc-90874fe5e68c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:04.209727 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04ad12-126d-42ff-a3cc-90874fe5e68c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:04.228759 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 107/198] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:59:04 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:59:04.229553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04ad12-126d-42ff-a3cc-90874fe5e68c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:04.246876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a3230-381a-4703-a49c-88d696c7f6ca None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:04.283048 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d1c0a-fdc2-44cc-995e-3e420ef7f13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:04.283733 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d1c0a-fdc2-44cc-995e-3e420ef7f13d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:04.284020 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d1c0a-fdc2-44cc-995e-3e420ef7f13d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:04.284347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d1c0a-fdc2-44cc-995e-3e420ef7f13d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:04.285245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d1c0a-fdc2-44cc-995e-3e420ef7f13d None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=36b45cb287dd49428bc291b9834d0b1e, user.id=588a9fdcdd234ceab4e72c361de4360e) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:04.285561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d1c0a-fdc2-44cc-995e-3e420ef7f13d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:04.287180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d1c0a-fdc2-44cc-995e-3e420ef7f13d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:04.293407 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 92/199] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:04 2022] GET /identity/v3/role_assignments?user.id=588a9fdcdd234ceab4e72c361de4360e&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=c184a177010144429f495f3290d28221 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:04.294847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3ad18-dd83-47a9-aed3-33f8ccf7f545 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:04.295143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3ad18-dd83-47a9-aed3-33f8ccf7f545 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:04.295419 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3ad18-dd83-47a9-aed3-33f8ccf7f545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:04.318532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2fe3ad18-dd83-47a9-aed3-33f8ccf7f545 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:04.320184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2fe3ad18-dd83-47a9-aed3-33f8ccf7f545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:04.333915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:04.370462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04ad12-126d-42ff-a3cc-90874fe5e68c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:04.405149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:04.409111 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 93/200] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:04.419133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edabf8-bbda-4f89-9c1b-e3650259ea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:04.420615 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edabf8-bbda-4f89-9c1b-e3650259ea90 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:04.420904 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edabf8-bbda-4f89-9c1b-e3650259ea90 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:04.421179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edabf8-bbda-4f89-9c1b-e3650259ea90 None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:04.423881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edabf8-bbda-4f89-9c1b-e3650259ea90 None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:04.424005 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edabf8-bbda-4f89-9c1b-e3650259ea90 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:04.425922 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 108/201] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:04 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:04.432327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3ad18-dd83-47a9-aed3-33f8ccf7f545 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:04.432781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edabf8-bbda-4f89-9c1b-e3650259ea90 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:04.555895 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3ad18-dd83-47a9-aed3-33f8ccf7f545 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:04.562133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edabf8-bbda-4f89-9c1b-e3650259ea90 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:04.601102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852e81e-8924-43ea-af10-46c4d7215c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:04.602748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e81e-8924-43ea-af10-46c4d7215c02 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:04.603724 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e81e-8924-43ea-af10-46c4d7215c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:04.604175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e81e-8924-43ea-af10-46c4d7215c02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:04.604956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b789ed-3823-4480-bdce-5603b98b2fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:04.606024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e81e-8924-43ea-af10-46c4d7215c02 None admin] RBAC: Adding query filter params (role.id=70d38d0a968a4e9585c5873d24cfbbb1, scope.project.id=36b45cb287dd49428bc291b9834d0b1e, user.id=588a9fdcdd234ceab4e72c361de4360e) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:04.606492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e81e-8924-43ea-af10-46c4d7215c02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:04.607214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b789ed-3823-4480-bdce-5603b98b2fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:04.607703 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b789ed-3823-4480-bdce-5603b98b2fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:04.608006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e81e-8924-43ea-af10-46c4d7215c02 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:04.608440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b789ed-3823-4480-bdce-5603b98b2fb9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:04.618261 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 94/202] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:04 2022] GET /identity/v3/role_assignments?user.id=588a9fdcdd234ceab4e72c361de4360e&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=70d38d0a968a4e9585c5873d24cfbbb1 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:04.628576 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b789ed-3823-4480-bdce-5603b98b2fb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-55b789ed-3823-4480-bdce-5603b98b2fb9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:04.636830 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:04.637796 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:04.638626 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 109/203] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:04.644714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852e81e-8924-43ea-af10-46c4d7215c02 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:04.657758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:04.757074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852e81e-8924-43ea-af10-46c4d7215c02 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:04.793719 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5309722-3a77-471a-b4b3-ee9eed91b2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:04.794816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5309722-3a77-471a-b4b3-ee9eed91b2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:04.795244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5309722-3a77-471a-b4b3-ee9eed91b2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:04.795659 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5309722-3a77-471a-b4b3-ee9eed91b2a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:04.796565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5309722-3a77-471a-b4b3-ee9eed91b2a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:04.797039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5309722-3a77-471a-b4b3-ee9eed91b2a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:04.798605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5309722-3a77-471a-b4b3-ee9eed91b2a0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:04.808441 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 95/204] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:04.814019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b789ed-3823-4480-bdce-5603b98b2fb9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:04.877879 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:04.931029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b789ed-3823-4480-bdce-5603b98b2fb9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:04.976856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80966193-643a-4b1c-be60-7d5224f6e548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:04.978060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80966193-643a-4b1c-be60-7d5224f6e548 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:04.978526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80966193-643a-4b1c-be60-7d5224f6e548 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:04.978940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80966193-643a-4b1c-be60-7d5224f6e548 None admin] PATH_INFO: `/v3/users/157cdb8bd93f4162b380bbe5ab80c179` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:04.980366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80966193-643a-4b1c-be60-7d5224f6e548 None admin] RBAC: Authorizing `identity:get_user(user_id=157cdb8bd93f4162b380bbe5ab80c179)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:04.982000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80966193-643a-4b1c-be60-7d5224f6e548 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:04.983874 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 110/205] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:04 2022] GET /identity/v3/users/157cdb8bd93f4162b380bbe5ab80c179 => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:04.993262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5309722-3a77-471a-b4b3-ee9eed91b2a0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:05.074114 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:05.105653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5309722-3a77-471a-b4b3-ee9eed91b2a0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:05.142937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c677f2-b504-499d-8da6-e9ccc9a9e9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:05.143859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c677f2-b504-499d-8da6-e9ccc9a9e9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:05.144287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c677f2-b504-499d-8da6-e9ccc9a9e9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:05.144706 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c677f2-b504-499d-8da6-e9ccc9a9e9b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:05.232353 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c677f2-b504-499d-8da6-e9ccc9a9e9b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:05.234062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c677f2-b504-499d-8da6-e9ccc9a9e9b7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:05.316936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:05.356706 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 96/206] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:05.362532 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80966193-643a-4b1c-be60-7d5224f6e548 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:05.465741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80966193-643a-4b1c-be60-7d5224f6e548 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:05.488136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:05.500236 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1823826-45a2-48b1-a263-e2652b78afe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:05.501130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823826-45a2-48b1-a263-e2652b78afe6 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:05.501412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823826-45a2-48b1-a263-e2652b78afe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:05.501694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823826-45a2-48b1-a263-e2652b78afe6 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:05.503073 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1823826-45a2-48b1-a263-e2652b78afe6 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:05.504564 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1823826-45a2-48b1-a263-e2652b78afe6 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:05.506731 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 111/207] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:05 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:05.515356 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c677f2-b504-499d-8da6-e9ccc9a9e9b7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:05.637990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c677f2-b504-499d-8da6-e9ccc9a9e9b7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:05.671949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a409c1-8694-4473-987b-3c9b3b39bbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:05.672632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a409c1-8694-4473-987b-3c9b3b39bbfd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:05.672956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a409c1-8694-4473-987b-3c9b3b39bbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:05.673235 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a409c1-8694-4473-987b-3c9b3b39bbfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:05.674081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a409c1-8694-4473-987b-3c9b3b39bbfd None admin] RBAC: Adding query filter params (role.id=8646f386c6704cb190f12a4b4e1a2245, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4, user.id=157cdb8bd93f4162b380bbe5ab80c179) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:05.674613 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a409c1-8694-4473-987b-3c9b3b39bbfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:05.676309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a409c1-8694-4473-987b-3c9b3b39bbfd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:05.682662 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 97/208] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:05 2022] GET /identity/v3/role_assignments?user.id=157cdb8bd93f4162b380bbe5ab80c179&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=8646f386c6704cb190f12a4b4e1a2245 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:06.692433 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf166372-b148-4162-ac75-286c12a2442f None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:06.693844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf166372-b148-4162-ac75-286c12a2442f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:06.694379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf166372-b148-4162-ac75-286c12a2442f None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:06.696272 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 112/209] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:06.701847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635ae6f-6537-49da-8730-c4e3c80357d1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:06.702735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635ae6f-6537-49da-8730-c4e3c80357d1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:06.703200 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635ae6f-6537-49da-8730-c4e3c80357d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:06.720644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c635ae6f-6537-49da-8730-c4e3c80357d1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:06.722093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c635ae6f-6537-49da-8730-c4e3c80357d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:06.749784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:06.803496 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 98/210] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:06.807705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771c561-521c-45b2-af34-740d523b91f4 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:06.808001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771c561-521c-45b2-af34-740d523b91f4 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:06.808282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771c561-521c-45b2-af34-740d523b91f4 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:06.809676 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 113/211] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:06.814886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c37ef-c356-4a88-9d3c-e25c42859840 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:06.815176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c37ef-c356-4a88-9d3c-e25c42859840 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:06.815454 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c37ef-c356-4a88-9d3c-e25c42859840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:06.828370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771c561-521c-45b2-af34-740d523b91f4 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:06.833588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8e3c37ef-c356-4a88-9d3c-e25c42859840 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:06.834807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8e3c37ef-c356-4a88-9d3c-e25c42859840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:06.906816 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 99/212] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:06.929490 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c37ef-c356-4a88-9d3c-e25c42859840 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:06.936923 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771c561-521c-45b2-af34-740d523b91f4 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:06.972638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610da664-8b63-49ca-9fb0-a5da6fb91739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:06.973381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610da664-8b63-49ca-9fb0-a5da6fb91739 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:06.973661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610da664-8b63-49ca-9fb0-a5da6fb91739 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:06.973938 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610da664-8b63-49ca-9fb0-a5da6fb91739 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:06.985078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610da664-8b63-49ca-9fb0-a5da6fb91739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-610da664-8b63-49ca-9fb0-a5da6fb91739 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:06.992956 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:06.993881 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:06.994719 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 114/213] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:59:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:06.995768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b04e4b-12d6-4ab7-9fa5-b63db163bd1a None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:06.995852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b04e4b-12d6-4ab7-9fa5-b63db163bd1a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:06.996079 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b04e4b-12d6-4ab7-9fa5-b63db163bd1a None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:06.997398 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 115/214] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:07.000175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc1938-cec1-49d9-ad6c-3c1061ad457c None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.000471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc1938-cec1-49d9-ad6c-3c1061ad457c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.001020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc1938-cec1-49d9-ad6c-3c1061ad457c None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.002695 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 116/215] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:07.003910 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc1938-cec1-49d9-ad6c-3c1061ad457c None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:07.033755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:07.041335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c37ef-c356-4a88-9d3c-e25c42859840 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.058140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:07.074355 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b803a6-9b8e-431e-acc5-4870ffd8e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.075164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b803a6-9b8e-431e-acc5-4870ffd8e490 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.075445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b803a6-9b8e-431e-acc5-4870ffd8e490 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.075719 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b803a6-9b8e-431e-acc5-4870ffd8e490 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.085918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b803a6-9b8e-431e-acc5-4870ffd8e490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-04b803a6-9b8e-431e-acc5-4870ffd8e490 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:07.093532 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:07.094541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:07.096002 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 100/216] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:07.099713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed366b3c-3a16-483c-98f2-3b7d5faa4471 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.100007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed366b3c-3a16-483c-98f2-3b7d5faa4471 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.100292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed366b3c-3a16-483c-98f2-3b7d5faa4471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.108590 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc1938-cec1-49d9-ad6c-3c1061ad457c None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.116895 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ed366b3c-3a16-483c-98f2-3b7d5faa4471 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:07.118074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ed366b3c-3a16-483c-98f2-3b7d5faa4471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:07.142259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff31787-b83b-4ffb-8403-5c45983b2b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.142935 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff31787-b83b-4ffb-8403-5c45983b2b2a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.143215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff31787-b83b-4ffb-8403-5c45983b2b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.143493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff31787-b83b-4ffb-8403-5c45983b2b2a None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.144237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff31787-b83b-4ffb-8403-5c45983b2b2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:07.144539 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff31787-b83b-4ffb-8403-5c45983b2b2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:07.145929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff31787-b83b-4ffb-8403-5c45983b2b2a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:07.154374 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 117/217] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:07.155534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b2f0b-5a06-4cb3-8253-087bf8ba9799 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.155819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b2f0b-5a06-4cb3-8253-087bf8ba9799 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.156109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b2f0b-5a06-4cb3-8253-087bf8ba9799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.171499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e95b2f0b-5a06-4cb3-8253-087bf8ba9799 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:07.172661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e95b2f0b-5a06-4cb3-8253-087bf8ba9799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:07.186045 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 101/218] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:07.191116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed366b3c-3a16-483c-98f2-3b7d5faa4471 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:07.237268 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 118/219] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:07.238021 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b2f0b-5a06-4cb3-8253-087bf8ba9799 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:07.278696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:07.279657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:07.295013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed366b3c-3a16-483c-98f2-3b7d5faa4471 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.327696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36692d-5dbf-45e4-90c2-c4460732d810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.328383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36692d-5dbf-45e4-90c2-c4460732d810 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.328668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36692d-5dbf-45e4-90c2-c4460732d810 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.328947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36692d-5dbf-45e4-90c2-c4460732d810 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.329711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36692d-5dbf-45e4-90c2-c4460732d810 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:07.330018 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36692d-5dbf-45e4-90c2-c4460732d810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:07.331478 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36692d-5dbf-45e4-90c2-c4460732d810 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:07.334056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b2f0b-5a06-4cb3-8253-087bf8ba9799 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.340083 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 102/220] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:07.343666 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36692d-5dbf-45e4-90c2-c4460732d810 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:07.368779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0ce82-b3de-4505-b99c-da72860c4004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.369696 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0ce82-b3de-4505-b99c-da72860c4004 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.369981 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0ce82-b3de-4505-b99c-da72860c4004 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.370283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0ce82-b3de-4505-b99c-da72860c4004 None admin] PATH_INFO: `/v3/users/588a9fdcdd234ceab4e72c361de4360e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.371523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0ce82-b3de-4505-b99c-da72860c4004 None admin] RBAC: Authorizing `identity:get_user(user_id=588a9fdcdd234ceab4e72c361de4360e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:07.373026 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0ce82-b3de-4505-b99c-da72860c4004 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:07.374741 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 119/221] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:07 2022] GET /identity/v3/users/588a9fdcdd234ceab4e72c361de4360e => generated 283 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:07.375512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0ce82-b3de-4505-b99c-da72860c4004 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:07.449105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36692d-5dbf-45e4-90c2-c4460732d810 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.475361 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0ce82-b3de-4505-b99c-da72860c4004 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.481993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d675d2-623f-4eba-b80f-66eb4d0ce92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.482839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d675d2-623f-4eba-b80f-66eb4d0ce92b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.483124 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d675d2-623f-4eba-b80f-66eb4d0ce92b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.483401 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d675d2-623f-4eba-b80f-66eb4d0ce92b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.485652 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:07.493699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d675d2-623f-4eba-b80f-66eb4d0ce92b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-26d675d2-623f-4eba-b80f-66eb4d0ce92b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:07.501767 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:07.502701 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:07.503515 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 103/222] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:59:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:07.504355 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d675d2-623f-4eba-b80f-66eb4d0ce92b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:07.513259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:07.515630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518d4d0-1ca8-4e41-b393-08af7535f49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.516452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518d4d0-1ca8-4e41-b393-08af7535f49c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.516730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518d4d0-1ca8-4e41-b393-08af7535f49c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.517004 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518d4d0-1ca8-4e41-b393-08af7535f49c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.518192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518d4d0-1ca8-4e41-b393-08af7535f49c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:07.519769 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518d4d0-1ca8-4e41-b393-08af7535f49c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:07.521407 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 120/223] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:07.522140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518d4d0-1ca8-4e41-b393-08af7535f49c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:07.604053 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d675d2-623f-4eba-b80f-66eb4d0ce92b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.617219 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518d4d0-1ca8-4e41-b393-08af7535f49c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.636512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c93df1-6fa0-439a-9c51-26d84f07c532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.637398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93df1-6fa0-439a-9c51-26d84f07c532 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.637682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93df1-6fa0-439a-9c51-26d84f07c532 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.637956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93df1-6fa0-439a-9c51-26d84f07c532 None admin] PATH_INFO: `/v3/users/2c4358ea27094a2aacf8562effae67ee` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.649799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c10cd-7e8d-4da3-af41-a1e898b9fdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.650706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c10cd-7e8d-4da3-af41-a1e898b9fdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.650983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c10cd-7e8d-4da3-af41-a1e898b9fdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.651269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c10cd-7e8d-4da3-af41-a1e898b9fdb8 None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.651384 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c93df1-6fa0-439a-9c51-26d84f07c532 None admin] RBAC: Authorizing `identity:get_user(user_id=2c4358ea27094a2aacf8562effae67ee)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:07.652467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c10cd-7e8d-4da3-af41-a1e898b9fdb8 None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:07.652778 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c93df1-6fa0-439a-9c51-26d84f07c532 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:07.653912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c10cd-7e8d-4da3-af41-a1e898b9fdb8 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:07.654414 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 104/224] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:07 2022] GET /identity/v3/users/2c4358ea27094a2aacf8562effae67ee => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:07.660336 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 121/225] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:07 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:07.660336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c10cd-7e8d-4da3-af41-a1e898b9fdb8 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:07.660533 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c93df1-6fa0-439a-9c51-26d84f07c532 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:07.704799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:07.718573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:07.755319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c93df1-6fa0-439a-9c51-26d84f07c532 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.758631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c10cd-7e8d-4da3-af41-a1e898b9fdb8 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.792658 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a0a47-849d-4ba3-b326-22e1980c8fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.793327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a0a47-849d-4ba3-b326-22e1980c8fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.793612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a0a47-849d-4ba3-b326-22e1980c8fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.793889 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a0a47-849d-4ba3-b326-22e1980c8fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.794694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a0a47-849d-4ba3-b326-22e1980c8fc1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:07.794831 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2b740-b617-4442-8cea-5a3cb2e3a82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.795003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a0a47-849d-4ba3-b326-22e1980c8fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:07.795510 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2b740-b617-4442-8cea-5a3cb2e3a82d None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.795792 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2b740-b617-4442-8cea-5a3cb2e3a82d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.796064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2b740-b617-4442-8cea-5a3cb2e3a82d None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.796433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a0a47-849d-4ba3-b326-22e1980c8fc1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:07.796778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2b740-b617-4442-8cea-5a3cb2e3a82d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:07.798235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2b740-b617-4442-8cea-5a3cb2e3a82d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:07.804618 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 105/226] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:07.805419 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a0a47-849d-4ba3-b326-22e1980c8fc1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:07.822978 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 122/227] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:59:07 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:59:07.824044 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2b740-b617-4442-8cea-5a3cb2e3a82d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:07.922520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a0a47-849d-4ba3-b326-22e1980c8fc1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.940801 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2b740-b617-4442-8cea-5a3cb2e3a82d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:07.943104 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:07.953430 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:07.964245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea58b8-b86d-4153-b72c-396a8eaf0058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.964932 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea58b8-b86d-4153-b72c-396a8eaf0058 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.965219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea58b8-b86d-4153-b72c-396a8eaf0058 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.965500 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea58b8-b86d-4153-b72c-396a8eaf0058 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.991308 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea271b-70b5-43dc-b9f4-71c8c835265d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:07.992227 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea271b-70b5-43dc-b9f4-71c8c835265d None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:07.992518 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea271b-70b5-43dc-b9f4-71c8c835265d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:07.992791 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea271b-70b5-43dc-b9f4-71c8c835265d None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e/users/588a9fdcdd234ceab4e72c361de4360e/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:07.995190 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea271b-70b5-43dc-b9f4-71c8c835265d None admin] RBAC: Authorizing `identity:create_grant(project_id=36b45cb287dd49428bc291b9834d0b1e, user_id=588a9fdcdd234ceab4e72c361de4360e, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:07.996707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea271b-70b5-43dc-b9f4-71c8c835265d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:08.013782 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 123/228] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:59:07 2022] PUT /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e/users/588a9fdcdd234ceab4e72c361de4360e/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:08.014944 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea271b-70b5-43dc-b9f4-71c8c835265d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:08.053983 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea58b8-b86d-4153-b72c-396a8eaf0058 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:08.055848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea58b8-b86d-4153-b72c-396a8eaf0058 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:08.164729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea271b-70b5-43dc-b9f4-71c8c835265d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:08.195996 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 106/229] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:08.203518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea58b8-b86d-4153-b72c-396a8eaf0058 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:08.207902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17b9c0-48cf-4dc2-afd1-11883ca4aa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:08.208950 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17b9c0-48cf-4dc2-afd1-11883ca4aa5b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:08.209244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17b9c0-48cf-4dc2-afd1-11883ca4aa5b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:08.209620 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17b9c0-48cf-4dc2-afd1-11883ca4aa5b None admin] PATH_INFO: `/v3/users/588a9fdcdd234ceab4e72c361de4360e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:08.210817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17b9c0-48cf-4dc2-afd1-11883ca4aa5b None admin] RBAC: Authorizing `identity:get_user(user_id=588a9fdcdd234ceab4e72c361de4360e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:08.212337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17b9c0-48cf-4dc2-afd1-11883ca4aa5b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:08.214031 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 124/230] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:08 2022] GET /identity/v3/users/588a9fdcdd234ceab4e72c361de4360e => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:08.222650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17b9c0-48cf-4dc2-afd1-11883ca4aa5b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:08.244383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:08.261774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:08.331867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea58b8-b86d-4153-b72c-396a8eaf0058 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:08.347860 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17b9c0-48cf-4dc2-afd1-11883ca4aa5b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:08.369605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413fb18-8516-4935-a098-167900b9a48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:08.370870 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413fb18-8516-4935-a098-167900b9a48b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:08.371366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413fb18-8516-4935-a098-167900b9a48b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:08.371828 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413fb18-8516-4935-a098-167900b9a48b None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:08.373338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413fb18-8516-4935-a098-167900b9a48b None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:08.375140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413fb18-8516-4935-a098-167900b9a48b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:08.377166 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 107/231] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:08 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:08.383169 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413fb18-8516-4935-a098-167900b9a48b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:08.391698 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b9b8c-d636-4a0e-aa5b-559aa91fb043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:08.392613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b9b8c-d636-4a0e-aa5b-559aa91fb043 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:08.392899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b9b8c-d636-4a0e-aa5b-559aa91fb043 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:08.393182 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b9b8c-d636-4a0e-aa5b-559aa91fb043 None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:08.394435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b9b8c-d636-4a0e-aa5b-559aa91fb043 None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:08.396503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b9b8c-d636-4a0e-aa5b-559aa91fb043 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:08.402301 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 125/232] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:08 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:08.406273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b9b8c-d636-4a0e-aa5b-559aa91fb043 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:08.561196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:08.572182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413fb18-8516-4935-a098-167900b9a48b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:08.600260 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:08.606920 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b9b8c-d636-4a0e-aa5b-559aa91fb043 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:08.608289 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e3dcb-4c5e-4e07-8999-38a1042d54e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:08.609042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e3dcb-4c5e-4e07-8999-38a1042d54e3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:08.609398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e3dcb-4c5e-4e07-8999-38a1042d54e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:08.609679 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e3dcb-4c5e-4e07-8999-38a1042d54e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:08.610635 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e3dcb-4c5e-4e07-8999-38a1042d54e3 None admin] RBAC: Adding query filter params (role.id=b10cd0627cc84a9c92dba333901e60e1, scope.project.id=36b45cb287dd49428bc291b9834d0b1e, user.id=2c4358ea27094a2aacf8562effae67ee) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:08.610953 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e3dcb-4c5e-4e07-8999-38a1042d54e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:08.612416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e3dcb-4c5e-4e07-8999-38a1042d54e3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:08.624441 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 108/233] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:08 2022] GET /identity/v3/role_assignments?user.id=2c4358ea27094a2aacf8562effae67ee&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:08.652634 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aafc93-c0e7-4460-867d-313c636500de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:08.653564 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aafc93-c0e7-4460-867d-313c636500de None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:08.653854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aafc93-c0e7-4460-867d-313c636500de None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:08.654133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aafc93-c0e7-4460-867d-313c636500de None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e/users/588a9fdcdd234ceab4e72c361de4360e/roles/70d38d0a968a4e9585c5873d24cfbbb1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:08.658058 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aafc93-c0e7-4460-867d-313c636500de None admin] RBAC: Authorizing `identity:create_grant(project_id=36b45cb287dd49428bc291b9834d0b1e, user_id=588a9fdcdd234ceab4e72c361de4360e, role_id=70d38d0a968a4e9585c5873d24cfbbb1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:08.659579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aafc93-c0e7-4460-867d-313c636500de None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:08.687579 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 126/234] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:59:08 2022] PUT /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e/users/588a9fdcdd234ceab4e72c361de4360e/roles/70d38d0a968a4e9585c5873d24cfbbb1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 09:59:10.283582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474585-269e-469f-b925-fa4a1d5d690e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:10.284131 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474585-269e-469f-b925-fa4a1d5d690e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:10.284131 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474585-269e-469f-b925-fa4a1d5d690e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:10.285773 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 109/235] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:10.302893 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90525495-a9ee-43e7-8ac4-9e5055774396 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:10.303351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90525495-a9ee-43e7-8ac4-9e5055774396 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:10.303449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90525495-a9ee-43e7-8ac4-9e5055774396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:10.331252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-90525495-a9ee-43e7-8ac4-9e5055774396 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:10.332472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-90525495-a9ee-43e7-8ac4-9e5055774396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:10.416985 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 127/236] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:10.456816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb474585-269e-469f-b925-fa4a1d5d690e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:10.525940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a838d2-8123-4484-9a48-421444791228 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:10.526288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a838d2-8123-4484-9a48-421444791228 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:10.526521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a838d2-8123-4484-9a48-421444791228 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:10.527921 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 128/237] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:10.533876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b06a0-1ee6-4372-84e2-95f714fd793c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:10.534101 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b06a0-1ee6-4372-84e2-95f714fd793c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:10.534399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b06a0-1ee6-4372-84e2-95f714fd793c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:10.558071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-804b06a0-1ee6-4372-84e2-95f714fd793c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:10.559292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-804b06a0-1ee6-4372-84e2-95f714fd793c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:10.595383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb474585-269e-469f-b925-fa4a1d5d690e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:10.638364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46317f07-0507-432f-870b-052e5fd1e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:10.639769 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46317f07-0507-432f-870b-052e5fd1e9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:10.640612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46317f07-0507-432f-870b-052e5fd1e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:10.641300 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46317f07-0507-432f-870b-052e5fd1e9f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:10.642750 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46317f07-0507-432f-870b-052e5fd1e9f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:10.644464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46317f07-0507-432f-870b-052e5fd1e9f2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:10.644570 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 129/238] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:10.650307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c1f76-b6e2-4a9f-9ce6-aa9762eb87f0 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:10.650307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c1f76-b6e2-4a9f-9ce6-aa9762eb87f0 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:10.650307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c1f76-b6e2-4a9f-9ce6-aa9762eb87f0 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:10.650307 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 130/239] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:10.652410 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 110/240] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:10.661563 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c1f76-b6e2-4a9f-9ce6-aa9762eb87f0 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:10.668551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c76338-2341-4c00-a2d2-c346dd0abaef None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:10.669027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c76338-2341-4c00-a2d2-c346dd0abaef None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:10.669472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c76338-2341-4c00-a2d2-c346dd0abaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:10.687225 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:10.693352 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b3c76338-2341-4c00-a2d2-c346dd0abaef None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:10.695052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b3c76338-2341-4c00-a2d2-c346dd0abaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:10.718201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:10.778347 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 111/241] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:10.779640 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c76338-2341-4c00-a2d2-c346dd0abaef None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:10.807863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c1f76-b6e2-4a9f-9ce6-aa9762eb87f0 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:10.848291 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b208a-2b60-462d-bc83-6f213e1aae8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:10.849413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b208a-2b60-462d-bc83-6f213e1aae8b None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:10.849957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b208a-2b60-462d-bc83-6f213e1aae8b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:10.850482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b208a-2b60-462d-bc83-6f213e1aae8b None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:10.851422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b208a-2b60-462d-bc83-6f213e1aae8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:10.855418 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b208a-2b60-462d-bc83-6f213e1aae8b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:10.892294 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 131/242] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:59:10 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:59:10.892294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b208a-2b60-462d-bc83-6f213e1aae8b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:10.932205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c76338-2341-4c00-a2d2-c346dd0abaef None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:11.013654 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75045c6e-d031-447f-b99c-2d23b78e4d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:11.014416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75045c6e-d031-447f-b99c-2d23b78e4d56 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.014704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75045c6e-d031-447f-b99c-2d23b78e4d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.014981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75045c6e-d031-447f-b99c-2d23b78e4d56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.023583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:11.047220 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75045c6e-d031-447f-b99c-2d23b78e4d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:11.052979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b208a-2b60-462d-bc83-6f213e1aae8b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-75045c6e-d031-447f-b99c-2d23b78e4d56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:11.063669 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:11.064616 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:11.065438 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 112/243] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:11.070979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75045c6e-d031-447f-b99c-2d23b78e4d56 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:11.099543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebfa37-0d08-48eb-840a-2bb696d8da3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:11.101358 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfa37-0d08-48eb-840a-2bb696d8da3c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.101861 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfa37-0d08-48eb-840a-2bb696d8da3c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.102438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfa37-0d08-48eb-840a-2bb696d8da3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.115388 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebfa37-0d08-48eb-840a-2bb696d8da3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-b6ebfa37-0d08-48eb-840a-2bb696d8da3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:11.124867 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:11.125841 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:11.127062 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 132/244] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:59:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:11.128028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ded48-f932-4a31-bae1-bc77a6353f6b None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.128096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ded48-f932-4a31-bae1-bc77a6353f6b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.128375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ded48-f932-4a31-bae1-bc77a6353f6b None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.129862 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 133/245] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:11.133687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ded48-f932-4a31-bae1-bc77a6353f6b None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:11.134820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:11.246150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75045c6e-d031-447f-b99c-2d23b78e4d56 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:11.268983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ded48-f932-4a31-bae1-bc77a6353f6b None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:11.280923 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f2536-70dc-48e2-a912-ebf0513df7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:11.281621 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2536-70dc-48e2-a912-ebf0513df7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.281891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2536-70dc-48e2-a912-ebf0513df7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.282163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2536-70dc-48e2-a912-ebf0513df7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.282969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f2536-70dc-48e2-a912-ebf0513df7b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:11.283278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f2536-70dc-48e2-a912-ebf0513df7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:11.284672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f2536-70dc-48e2-a912-ebf0513df7b8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:11.298380 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 113/246] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:11.299668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded66403-0c20-4fa6-a8e9-d6fe0c6f6556 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.299958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded66403-0c20-4fa6-a8e9-d6fe0c6f6556 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.300241 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded66403-0c20-4fa6-a8e9-d6fe0c6f6556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.318578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ded66403-0c20-4fa6-a8e9-d6fe0c6f6556 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:11.319747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ded66403-0c20-4fa6-a8e9-d6fe0c6f6556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:11.332472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be5de4-6f42-4755-b5bf-0ea4465f1e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:11.333468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be5de4-6f42-4755-b5bf-0ea4465f1e15 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.333782 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be5de4-6f42-4755-b5bf-0ea4465f1e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.334054 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be5de4-6f42-4755-b5bf-0ea4465f1e15 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.334868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be5de4-6f42-4755-b5bf-0ea4465f1e15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:11.335625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be5de4-6f42-4755-b5bf-0ea4465f1e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:11.336548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be5de4-6f42-4755-b5bf-0ea4465f1e15 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:11.338517 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:11.347570 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 134/247] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:11.348438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be5de4-6f42-4755-b5bf-0ea4465f1e15 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:11.401320 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 114/248] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:11.402181 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded66403-0c20-4fa6-a8e9-d6fe0c6f6556 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:11.412607 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:11.464883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be5de4-6f42-4755-b5bf-0ea4465f1e15 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:11.509697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe7d46-7954-4be3-907a-10d1c954c970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:11.511593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe7d46-7954-4be3-907a-10d1c954c970 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.511593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe7d46-7954-4be3-907a-10d1c954c970 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.511798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe7d46-7954-4be3-907a-10d1c954c970 None admin] PATH_INFO: `/v3/users/dad4a61888834a5ea6c92cdc9d448a0e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.520775 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded66403-0c20-4fa6-a8e9-d6fe0c6f6556 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:11.536353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe7d46-7954-4be3-907a-10d1c954c970 None admin] RBAC: Authorizing `identity:get_user(user_id=dad4a61888834a5ea6c92cdc9d448a0e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:11.538670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe7d46-7954-4be3-907a-10d1c954c970 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:11.540708 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 135/249] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:11 2022] GET /identity/v3/users/dad4a61888834a5ea6c92cdc9d448a0e => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:11.542383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9bb92-cdea-4c94-9efb-9c675a49459e None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.542885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9bb92-cdea-4c94-9efb-9c675a49459e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.543629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9bb92-cdea-4c94-9efb-9c675a49459e None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.546308 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 136/250] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:11.547334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9bb92-cdea-4c94-9efb-9c675a49459e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:11.570893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e41e7a-d911-4554-a803-4157da648415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:11.572244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e41e7a-d911-4554-a803-4157da648415 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.572533 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e41e7a-d911-4554-a803-4157da648415 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.572811 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e41e7a-d911-4554-a803-4157da648415 None admin] PATH_INFO: `/v3/users/588a9fdcdd234ceab4e72c361de4360e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.574076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e41e7a-d911-4554-a803-4157da648415 None admin] RBAC: Authorizing `identity:get_user(user_id=588a9fdcdd234ceab4e72c361de4360e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:11.576143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e41e7a-d911-4554-a803-4157da648415 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:11.578349 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 115/251] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:11 2022] GET /identity/v3/users/588a9fdcdd234ceab4e72c361de4360e => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:11.583587 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e41e7a-d911-4554-a803-4157da648415 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:11.643637 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:11.663338 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9bb92-cdea-4c94-9efb-9c675a49459e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:11.674070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:11.699756 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b5638-79f6-4961-9c0c-f1b9d55542ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:11.700615 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b5638-79f6-4961-9c0c-f1b9d55542ab None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.700945 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b5638-79f6-4961-9c0c-f1b9d55542ab None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.701221 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b5638-79f6-4961-9c0c-f1b9d55542ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.702330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e41e7a-d911-4554-a803-4157da648415 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:11.713268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b5638-79f6-4961-9c0c-f1b9d55542ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-222b5638-79f6-4961-9c0c-f1b9d55542ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:11.721875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:11.722815 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:11.723621 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 137/252] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:59:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:11.724816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea381f5-2188-4ef4-a771-e7e81147aa5c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.724816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea381f5-2188-4ef4-a771-e7e81147aa5c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.724963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea381f5-2188-4ef4-a771-e7e81147aa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.742623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9c022-3a15-4bdf-8e70-6ba92244588d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:11.743672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9c022-3a15-4bdf-8e70-6ba92244588d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:11.744153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9c022-3a15-4bdf-8e70-6ba92244588d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:11.744629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9c022-3a15-4bdf-8e70-6ba92244588d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:11.748066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3ea381f5-2188-4ef4-a771-e7e81147aa5c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:11.749470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3ea381f5-2188-4ef4-a771-e7e81147aa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:11.815744 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9c022-3a15-4bdf-8e70-6ba92244588d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:11.817359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9c022-3a15-4bdf-8e70-6ba92244588d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:11.852072 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 138/253] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:11.854288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea381f5-2188-4ef4-a771-e7e81147aa5c None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:11.907576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:11.908815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:11.942581 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 116/254] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:11.944472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9c022-3a15-4bdf-8e70-6ba92244588d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:11.979525 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea381f5-2188-4ef4-a771-e7e81147aa5c None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:12.028162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0e93d-e035-4ba6-8fbe-3f67d70f0a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:12.029697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0e93d-e035-4ba6-8fbe-3f67d70f0a35 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:12.029977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0e93d-e035-4ba6-8fbe-3f67d70f0a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:12.030295 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0e93d-e035-4ba6-8fbe-3f67d70f0a35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:12.059971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9c022-3a15-4bdf-8e70-6ba92244588d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:12.095729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e9e57-f5aa-4e6e-9959-1fd7324a8f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:12.096528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e9e57-f5aa-4e6e-9959-1fd7324a8f2f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:12.096880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e9e57-f5aa-4e6e-9959-1fd7324a8f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:12.097226 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e9e57-f5aa-4e6e-9959-1fd7324a8f2f None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:12.098056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e9e57-f5aa-4e6e-9959-1fd7324a8f2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:12.098458 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e9e57-f5aa-4e6e-9959-1fd7324a8f2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:12.100006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e9e57-f5aa-4e6e-9959-1fd7324a8f2f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:12.101439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0e93d-e035-4ba6-8fbe-3f67d70f0a35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:12.102986 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0e93d-e035-4ba6-8fbe-3f67d70f0a35 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:12.108831 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 117/255] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:12.109836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e9e57-f5aa-4e6e-9959-1fd7324a8f2f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:12.150842 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:12.152474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:12.209675 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 139/256] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:12.210548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0e93d-e035-4ba6-8fbe-3f67d70f0a35 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:12.221067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e9e57-f5aa-4e6e-9959-1fd7324a8f2f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:12.257097 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844dabc5-5ae5-4c6f-85ae-c2e3d0519570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:12.257922 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dabc5-5ae5-4c6f-85ae-c2e3d0519570 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:12.258208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dabc5-5ae5-4c6f-85ae-c2e3d0519570 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:12.258529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dabc5-5ae5-4c6f-85ae-c2e3d0519570 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:12.269309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844dabc5-5ae5-4c6f-85ae-c2e3d0519570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-844dabc5-5ae5-4c6f-85ae-c2e3d0519570 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:12.279858 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:12.280752 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:12.281546 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:12.282636 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 118/257] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:12.286256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844dabc5-5ae5-4c6f-85ae-c2e3d0519570 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:12.312117 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0e93d-e035-4ba6-8fbe-3f67d70f0a35 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:12.349956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34560b10-73f3-4aac-a6ec-e6954741c7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:12.350982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34560b10-73f3-4aac-a6ec-e6954741c7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:12.351159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34560b10-73f3-4aac-a6ec-e6954741c7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:12.351405 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34560b10-73f3-4aac-a6ec-e6954741c7c2 None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:12.353116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34560b10-73f3-4aac-a6ec-e6954741c7c2 None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:12.354574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34560b10-73f3-4aac-a6ec-e6954741c7c2 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:12.361066 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 140/258] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:12 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:12.361066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34560b10-73f3-4aac-a6ec-e6954741c7c2 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:12.372319 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:12.422247 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:12.428745 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844dabc5-5ae5-4c6f-85ae-c2e3d0519570 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:12.470180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34560b10-73f3-4aac-a6ec-e6954741c7c2 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:12.477043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c316c8f-92b0-4b30-827c-4f91b774a135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:12.478688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c316c8f-92b0-4b30-827c-4f91b774a135 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:12.479169 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c316c8f-92b0-4b30-827c-4f91b774a135 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:12.479641 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c316c8f-92b0-4b30-827c-4f91b774a135 None admin] PATH_INFO: `/v3/users/588a9fdcdd234ceab4e72c361de4360e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:12.481862 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c316c8f-92b0-4b30-827c-4f91b774a135 None admin] RBAC: Authorizing `identity:get_user(user_id=588a9fdcdd234ceab4e72c361de4360e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:12.484430 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c316c8f-92b0-4b30-827c-4f91b774a135 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:12.486960 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 119/259] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:12 2022] GET /identity/v3/users/588a9fdcdd234ceab4e72c361de4360e => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:12.491674 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c316c8f-92b0-4b30-827c-4f91b774a135 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:12.516611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6dba98-8fd3-4a48-ba12-ff35b7a54f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:12.518167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dba98-8fd3-4a48-ba12-ff35b7a54f52 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:12.518908 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dba98-8fd3-4a48-ba12-ff35b7a54f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:12.519356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dba98-8fd3-4a48-ba12-ff35b7a54f52 None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:12.520785 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6dba98-8fd3-4a48-ba12-ff35b7a54f52 None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:12.523014 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6dba98-8fd3-4a48-ba12-ff35b7a54f52 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:12.525562 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 141/260] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:12 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:12.526485 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6dba98-8fd3-4a48-ba12-ff35b7a54f52 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:12.599530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c316c8f-92b0-4b30-827c-4f91b774a135 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:12.624495 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:12.645217 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4da1d5-4994-407b-aa25-7bf61b48c797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:12.645477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6dba98-8fd3-4a48-ba12-ff35b7a54f52 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:12.646346 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4da1d5-4994-407b-aa25-7bf61b48c797 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:12.646781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4da1d5-4994-407b-aa25-7bf61b48c797 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:12.647189 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4da1d5-4994-407b-aa25-7bf61b48c797 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:12.648949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4da1d5-4994-407b-aa25-7bf61b48c797 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:12.649404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4da1d5-4994-407b-aa25-7bf61b48c797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:12.650976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4da1d5-4994-407b-aa25-7bf61b48c797 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:12.661714 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 120/261] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:12.662886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4da1d5-4994-407b-aa25-7bf61b48c797 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:12.682282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86162fa7-6b2e-4e5a-b28b-0d508edd0de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:12.683075 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86162fa7-6b2e-4e5a-b28b-0d508edd0de0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:12.683461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86162fa7-6b2e-4e5a-b28b-0d508edd0de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:12.683825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86162fa7-6b2e-4e5a-b28b-0d508edd0de0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:12.684865 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86162fa7-6b2e-4e5a-b28b-0d508edd0de0 None admin] RBAC: Adding query filter params (role.id=8646f386c6704cb190f12a4b4e1a2245, scope.project.id=36b45cb287dd49428bc291b9834d0b1e, user.id=dad4a61888834a5ea6c92cdc9d448a0e) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:12.685321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86162fa7-6b2e-4e5a-b28b-0d508edd0de0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:12.687032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86162fa7-6b2e-4e5a-b28b-0d508edd0de0 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:12.698372 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 142/262] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:12 2022] GET /identity/v3/role_assignments?user.id=dad4a61888834a5ea6c92cdc9d448a0e&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=8646f386c6704cb190f12a4b4e1a2245 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:12.698372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86162fa7-6b2e-4e5a-b28b-0d508edd0de0 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:12.705978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:12.857848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86162fa7-6b2e-4e5a-b28b-0d508edd0de0 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:12.879821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4da1d5-4994-407b-aa25-7bf61b48c797 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:12.919871 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c739a-2612-48a0-a719-e3def36c9c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:12.921091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c739a-2612-48a0-a719-e3def36c9c8c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:12.921575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c739a-2612-48a0-a719-e3def36c9c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:12.922045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c739a-2612-48a0-a719-e3def36c9c8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:12.923140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c739a-2612-48a0-a719-e3def36c9c8c None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=36b45cb287dd49428bc291b9834d0b1e, user.id=588a9fdcdd234ceab4e72c361de4360e) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:12.923644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c739a-2612-48a0-a719-e3def36c9c8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:12.925669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c739a-2612-48a0-a719-e3def36c9c8c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:12.934814 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 143/263] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:12 2022] GET /identity/v3/role_assignments?user.id=588a9fdcdd234ceab4e72c361de4360e&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=c184a177010144429f495f3290d28221 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:12.937671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c739a-2612-48a0-a719-e3def36c9c8c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:12.958514 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:12.977829 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997c45b-edaf-4679-923e-b23a36740585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:12.978682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997c45b-edaf-4679-923e-b23a36740585 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:12.979012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997c45b-edaf-4679-923e-b23a36740585 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:12.979295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997c45b-edaf-4679-923e-b23a36740585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:13.057015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c739a-2612-48a0-a719-e3def36c9c8c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:13.097864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76010b74-95b0-4c36-8d61-b5f6d5731607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:13.099150 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76010b74-95b0-4c36-8d61-b5f6d5731607 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:13.099955 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76010b74-95b0-4c36-8d61-b5f6d5731607 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:13.100420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76010b74-95b0-4c36-8d61-b5f6d5731607 None admin] PATH_INFO: `/v3/users/2c4358ea27094a2aacf8562effae67ee` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:13.102356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76010b74-95b0-4c36-8d61-b5f6d5731607 None admin] RBAC: Authorizing `identity:get_user(user_id=2c4358ea27094a2aacf8562effae67ee)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:13.104013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76010b74-95b0-4c36-8d61-b5f6d5731607 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:13.105852 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 144/264] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:12 2022] GET /identity/v3/users/2c4358ea27094a2aacf8562effae67ee => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:13.115060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76010b74-95b0-4c36-8d61-b5f6d5731607 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:13.135987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997c45b-edaf-4679-923e-b23a36740585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:13.137649 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997c45b-edaf-4679-923e-b23a36740585 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:13.215678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:13.232917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:13.283846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76010b74-95b0-4c36-8d61-b5f6d5731607 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:13.357629 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 121/265] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:13.357629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94d349-a0d3-434b-908a-8cda1489b2ed None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:13.357629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94d349-a0d3-434b-908a-8cda1489b2ed None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:13.357629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94d349-a0d3-434b-908a-8cda1489b2ed None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:13.357629 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 122/266] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:13.360785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94d349-a0d3-434b-908a-8cda1489b2ed None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:13.361928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e51bfa-d619-4b78-82c8-2d73bc1e044c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:13.363403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e51bfa-d619-4b78-82c8-2d73bc1e044c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:13.363872 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e51bfa-d619-4b78-82c8-2d73bc1e044c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:13.364367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e51bfa-d619-4b78-82c8-2d73bc1e044c None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:13.368745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e51bfa-d619-4b78-82c8-2d73bc1e044c None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:13.370496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e51bfa-d619-4b78-82c8-2d73bc1e044c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:13.372503 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 145/267] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:13 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:13.373898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4cb69-aebf-4486-95a7-8ca747d9af16 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:13.374425 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4cb69-aebf-4486-95a7-8ca747d9af16 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:13.374900 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4cb69-aebf-4486-95a7-8ca747d9af16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:13.393499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-8de4cb69-aebf-4486-95a7-8ca747d9af16 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:13.395099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8de4cb69-aebf-4486-95a7-8ca747d9af16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:13.486062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94d349-a0d3-434b-908a-8cda1489b2ed None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:13.499367 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 146/268] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:13.499367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4cb69-aebf-4486-95a7-8ca747d9af16 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:13.529904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:13.539661 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89e7e1-89e7-49b9-be6b-b23c6adc81c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:13.540542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e7e1-89e7-49b9-be6b-b23c6adc81c8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:13.540834 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e7e1-89e7-49b9-be6b-b23c6adc81c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:13.541110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e7e1-89e7-49b9-be6b-b23c6adc81c8 None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:13.542334 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89e7e1-89e7-49b9-be6b-b23c6adc81c8 None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:13.543787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89e7e1-89e7-49b9-be6b-b23c6adc81c8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:13.545630 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 123/269] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:13 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:13.546751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5209e-53f4-4686-ac46-4205a750919d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:13.547039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5209e-53f4-4686-ac46-4205a750919d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:13.547314 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5209e-53f4-4686-ac46-4205a750919d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:13.552521 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 124/270] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:13.552521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5209e-53f4-4686-ac46-4205a750919d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:13.586288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:13.612944 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4cb69-aebf-4486-95a7-8ca747d9af16 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:13.662095 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c460d-ac3c-4536-b63c-4d4c52bd9370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:13.663008 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c460d-ac3c-4536-b63c-4d4c52bd9370 None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:13.663296 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c460d-ac3c-4536-b63c-4d4c52bd9370 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:13.663567 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c460d-ac3c-4536-b63c-4d4c52bd9370 None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e/users/2c4358ea27094a2aacf8562effae67ee/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:13.665677 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c460d-ac3c-4536-b63c-4d4c52bd9370 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b45cb287dd49428bc291b9834d0b1e, user_id=2c4358ea27094a2aacf8562effae67ee, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:13.667235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c460d-ac3c-4536-b63c-4d4c52bd9370 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:13.667485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5209e-53f4-4686-ac46-4205a750919d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:13.683002 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 147/271] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:59:13 2022] PUT /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e/users/2c4358ea27094a2aacf8562effae67ee/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:13.683826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c460d-ac3c-4536-b63c-4d4c52bd9370 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:13.713638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30879be2-38ce-494a-97f3-fd9534f4c3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:13.714887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30879be2-38ce-494a-97f3-fd9534f4c3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:13.715228 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30879be2-38ce-494a-97f3-fd9534f4c3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:13.715571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30879be2-38ce-494a-97f3-fd9534f4c3d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:13.728160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30879be2-38ce-494a-97f3-fd9534f4c3d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-30879be2-38ce-494a-97f3-fd9534f4c3d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:13.736265 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:13.737282 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:13.738063 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:13.742946 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 125/272] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:13.744567 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6587cc-eaab-47f3-9765-240908bb952c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:13.744948 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6587cc-eaab-47f3-9765-240908bb952c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:13.745284 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6587cc-eaab-47f3-9765-240908bb952c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:13.775125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dd6587cc-eaab-47f3-9765-240908bb952c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:13.776381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dd6587cc-eaab-47f3-9765-240908bb952c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:13.828582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c460d-ac3c-4536-b63c-4d4c52bd9370 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:13.848883 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:13.869100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f5153-17aa-4165-933a-464370214426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:13.869302 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f5153-17aa-4165-933a-464370214426 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:13.869540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f5153-17aa-4165-933a-464370214426 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:13.869815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f5153-17aa-4165-933a-464370214426 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:13.871432 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f5153-17aa-4165-933a-464370214426 None admin] RBAC: Adding query filter params (role.id=70d38d0a968a4e9585c5873d24cfbbb1, scope.project.id=36b45cb287dd49428bc291b9834d0b1e, user.id=588a9fdcdd234ceab4e72c361de4360e) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:13.871432 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f5153-17aa-4165-933a-464370214426 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:13.872102 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 126/273] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:13.872813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6587cc-eaab-47f3-9765-240908bb952c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:13.876881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f5153-17aa-4165-933a-464370214426 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:13.879039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:13.891011 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 148/274] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:13 2022] GET /identity/v3/role_assignments?user.id=588a9fdcdd234ceab4e72c361de4360e&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=70d38d0a968a4e9585c5873d24cfbbb1 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:13.905171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f5153-17aa-4165-933a-464370214426 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:14.013271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6587cc-eaab-47f3-9765-240908bb952c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:14.020102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f5153-17aa-4165-933a-464370214426 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:14.057403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace411b6-8a81-423e-929a-a25282a2448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:14.058246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace411b6-8a81-423e-929a-a25282a2448a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:14.058547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace411b6-8a81-423e-929a-a25282a2448a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:14.058824 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace411b6-8a81-423e-929a-a25282a2448a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:14.060062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace411b6-8a81-423e-929a-a25282a2448a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:14.061537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace411b6-8a81-423e-929a-a25282a2448a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:14.063270 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 149/275] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:14.069151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace411b6-8a81-423e-929a-a25282a2448a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:14.077400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb44f7-df6a-4f60-8f2f-87c8ed43fae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:14.078142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb44f7-df6a-4f60-8f2f-87c8ed43fae7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:14.078464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb44f7-df6a-4f60-8f2f-87c8ed43fae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:14.078744 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb44f7-df6a-4f60-8f2f-87c8ed43fae7 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:14.079549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb44f7-df6a-4f60-8f2f-87c8ed43fae7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:14.079866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb44f7-df6a-4f60-8f2f-87c8ed43fae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:14.082188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb44f7-df6a-4f60-8f2f-87c8ed43fae7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:14.104148 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 127/276] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:14.106043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb44f7-df6a-4f60-8f2f-87c8ed43fae7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:14.157325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:14.158581 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:14.199213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace411b6-8a81-423e-929a-a25282a2448a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:14.217486 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb44f7-df6a-4f60-8f2f-87c8ed43fae7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:14.239327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f84db-f339-4921-bdec-dbfcd5caef38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:14.241136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f84db-f339-4921-bdec-dbfcd5caef38 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:14.241634 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f84db-f339-4921-bdec-dbfcd5caef38 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:14.242157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f84db-f339-4921-bdec-dbfcd5caef38 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:14.243121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f84db-f339-4921-bdec-dbfcd5caef38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:14.244790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f84db-f339-4921-bdec-dbfcd5caef38 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:14.259190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb3ac8-6457-4f12-99ca-047bbe4e6133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:14.259506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb3ac8-6457-4f12-99ca-047bbe4e6133 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:14.259832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb3ac8-6457-4f12-99ca-047bbe4e6133 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:14.260094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb3ac8-6457-4f12-99ca-047bbe4e6133 None admin] PATH_INFO: `/v3/users/e0a959135b3542aab1864503c6d6a425` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:14.271728 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 150/277] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:59:14 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 09:59:14.278985 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb3ac8-6457-4f12-99ca-047bbe4e6133 None admin] RBAC: Authorizing `identity:get_user(user_id=e0a959135b3542aab1864503c6d6a425)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:14.280512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb3ac8-6457-4f12-99ca-047bbe4e6133 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:14.282439 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 128/278] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:14 2022] GET /identity/v3/users/e0a959135b3542aab1864503c6d6a425 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:14.288739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f84db-f339-4921-bdec-dbfcd5caef38 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:14.415809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f84db-f339-4921-bdec-dbfcd5caef38 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:14.444670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:14.460752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f105d19-8284-433b-ab3f-89ec7f3df98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:14.461433 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105d19-8284-433b-ab3f-89ec7f3df98c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:14.461719 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105d19-8284-433b-ab3f-89ec7f3df98c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:14.461997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105d19-8284-433b-ab3f-89ec7f3df98c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:14.462870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f105d19-8284-433b-ab3f-89ec7f3df98c None admin] RBAC: Adding query filter params (role.id=b10cd0627cc84a9c92dba333901e60e1, user.id=e0a959135b3542aab1864503c6d6a425, scope.system=all) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:14.463186 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f105d19-8284-433b-ab3f-89ec7f3df98c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:14.464680 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f105d19-8284-433b-ab3f-89ec7f3df98c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:14.473158 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 151/279] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Jul 5 09:59:14 2022] GET /identity/v3/role_assignments?user.id=e0a959135b3542aab1864503c6d6a425&scope.system=all&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:15.076284 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231749c3-eeae-44f5-a916-037722814782 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:15.076786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231749c3-eeae-44f5-a916-037722814782 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:15.076786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231749c3-eeae-44f5-a916-037722814782 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:15.078519 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 129/280] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:15.084554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d89cb4-4796-4d05-a2cb-7615e3ea0a86 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:15.084842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d89cb4-4796-4d05-a2cb-7615e3ea0a86 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:15.085123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d89cb4-4796-4d05-a2cb-7615e3ea0a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:15.107313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-14d89cb4-4796-4d05-a2cb-7615e3ea0a86 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:15.108889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-14d89cb4-4796-4d05-a2cb-7615e3ea0a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:15.179207 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 152/281] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:15.222600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231749c3-eeae-44f5-a916-037722814782 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:15.356101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231749c3-eeae-44f5-a916-037722814782 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:15.364153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:15.401818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2531b18-7aab-4ce2-8bf0-9095faa518cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:15.402740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2531b18-7aab-4ce2-8bf0-9095faa518cc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:15.403029 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2531b18-7aab-4ce2-8bf0-9095faa518cc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:15.403350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2531b18-7aab-4ce2-8bf0-9095faa518cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:15.415787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2531b18-7aab-4ce2-8bf0-9095faa518cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e2531b18-7aab-4ce2-8bf0-9095faa518cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:15.427572 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:15.428464 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:15.429255 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 130/282] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:15.434958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d89cb4-4796-4d05-a2cb-7615e3ea0a86 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:15.563076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d89cb4-4796-4d05-a2cb-7615e3ea0a86 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:15.606682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:15.613833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c9ac8-8202-464e-9a89-9dac57de5810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:15.614573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c9ac8-8202-464e-9a89-9dac57de5810 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:15.614823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c9ac8-8202-464e-9a89-9dac57de5810 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:15.615120 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c9ac8-8202-464e-9a89-9dac57de5810 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:15.615950 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c9ac8-8202-464e-9a89-9dac57de5810 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:15.616263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c9ac8-8202-464e-9a89-9dac57de5810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:15.617728 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c9ac8-8202-464e-9a89-9dac57de5810 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:15.632086 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2531b18-7aab-4ce2-8bf0-9095faa518cc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:15.633097 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 153/283] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:15.809555 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2531b18-7aab-4ce2-8bf0-9095faa518cc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:15.855801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d44d2-2407-4021-a9d7-c33fa951e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:15.856815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d44d2-2407-4021-a9d7-c33fa951e64a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:15.857046 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d44d2-2407-4021-a9d7-c33fa951e64a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:15.857327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d44d2-2407-4021-a9d7-c33fa951e64a None admin] PATH_INFO: `/v3/users/dad4a61888834a5ea6c92cdc9d448a0e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:15.858612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d44d2-2407-4021-a9d7-c33fa951e64a None admin] RBAC: Authorizing `identity:get_user(user_id=dad4a61888834a5ea6c92cdc9d448a0e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:15.860095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d44d2-2407-4021-a9d7-c33fa951e64a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:15.862048 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 131/284] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:15 2022] GET /identity/v3/users/dad4a61888834a5ea6c92cdc9d448a0e => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:15.867542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c9ac8-8202-464e-9a89-9dac57de5810 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:15.986537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c9ac8-8202-464e-9a89-9dac57de5810 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:16.033708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83eb822-eee9-4f9e-849c-55490e32d11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:16.034662 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eb822-eee9-4f9e-849c-55490e32d11c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:16.034944 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eb822-eee9-4f9e-849c-55490e32d11c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:16.035219 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eb822-eee9-4f9e-849c-55490e32d11c None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:16.036431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83eb822-eee9-4f9e-849c-55490e32d11c None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:16.037985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83eb822-eee9-4f9e-849c-55490e32d11c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:16.039726 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 154/285] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:15 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:16.045541 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d44d2-2407-4021-a9d7-c33fa951e64a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:16.089840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:16.178334 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d44d2-2407-4021-a9d7-c33fa951e64a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:16.195041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455d739-dc5e-47dc-8534-6a22e5863fa1 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:16.195041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455d739-dc5e-47dc-8534-6a22e5863fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:16.195041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455d739-dc5e-47dc-8534-6a22e5863fa1 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:16.196102 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 155/286] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 09:59:16.203673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36279d67-3fec-4e5d-8eec-040aafed16d8 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:16.204039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36279d67-3fec-4e5d-8eec-040aafed16d8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:16.204381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36279d67-3fec-4e5d-8eec-040aafed16d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:16.223399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4f2d8-9393-49e6-857b-8ae88cfa18df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:16.224411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4f2d8-9393-49e6-857b-8ae88cfa18df None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:16.224704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4f2d8-9393-49e6-857b-8ae88cfa18df None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:16.224989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4f2d8-9393-49e6-857b-8ae88cfa18df None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e/users/dad4a61888834a5ea6c92cdc9d448a0e/roles/8646f386c6704cb190f12a4b4e1a2245` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:16.227085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4f2d8-9393-49e6-857b-8ae88cfa18df None admin] RBAC: Authorizing `identity:create_grant(project_id=36b45cb287dd49428bc291b9834d0b1e, user_id=dad4a61888834a5ea6c92cdc9d448a0e, role_id=8646f386c6704cb190f12a4b4e1a2245)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:16.228676 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4f2d8-9393-49e6-857b-8ae88cfa18df None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:16.236335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-36279d67-3fec-4e5d-8eec-040aafed16d8 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:16.237555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-36279d67-3fec-4e5d-8eec-040aafed16d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:16.249185 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 132/287] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:59:16 2022] PUT /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e/users/dad4a61888834a5ea6c92cdc9d448a0e/roles/8646f386c6704cb190f12a4b4e1a2245 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:16.300585 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:16.335463 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 156/288] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:16.359294 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4f2d8-9393-49e6-857b-8ae88cfa18df None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:16.483225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:16.491242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4f2d8-9393-49e6-857b-8ae88cfa18df None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:16.528476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75b5dc-40f9-4b3b-9582-eed2313b7db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:16.529308 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75b5dc-40f9-4b3b-9582-eed2313b7db7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:16.529593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75b5dc-40f9-4b3b-9582-eed2313b7db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:16.529867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75b5dc-40f9-4b3b-9582-eed2313b7db7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:16.541483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75b5dc-40f9-4b3b-9582-eed2313b7db7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ac75b5dc-40f9-4b3b-9582-eed2313b7db7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:16.549417 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:16.550316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:16.551103 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:16.554329 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 133/289] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:16.557024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36279d67-3fec-4e5d-8eec-040aafed16d8 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:16.670982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36279d67-3fec-4e5d-8eec-040aafed16d8 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:16.710378 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2d143-3a21-49c4-8661-4ca9a8fb031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:16.711092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2d143-3a21-49c4-8661-4ca9a8fb031b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:16.711377 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2d143-3a21-49c4-8661-4ca9a8fb031b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:16.711651 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2d143-3a21-49c4-8661-4ca9a8fb031b None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:16.712435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2d143-3a21-49c4-8661-4ca9a8fb031b None admin] RBAC: Adding query filter params (name=member) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:16.712744 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2d143-3a21-49c4-8661-4ca9a8fb031b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:16.714145 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2d143-3a21-49c4-8661-4ca9a8fb031b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:16.727344 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 157/290] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:16.729178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75b5dc-40f9-4b3b-9582-eed2313b7db7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:16.888818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75b5dc-40f9-4b3b-9582-eed2313b7db7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:16.898373 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1f98b-6386-4ca9-a573-d4e3518538d2 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:16.898660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1f98b-6386-4ca9-a573-d4e3518538d2 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:16.898933 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1f98b-6386-4ca9-a573-d4e3518538d2 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:16.900349 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 158/291] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 09:59:16.905673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b2a32-7294-47c5-9437-36a03a57053c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:16.905960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b2a32-7294-47c5-9437-36a03a57053c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:16.906254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b2a32-7294-47c5-9437-36a03a57053c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:16.936578 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-084b2a32-7294-47c5-9437-36a03a57053c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:16.937783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-084b2a32-7294-47c5-9437-36a03a57053c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:16.960833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad584f-c7b2-473f-acf8-f45e8b101512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:16.961780 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad584f-c7b2-473f-acf8-f45e8b101512 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:16.962056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad584f-c7b2-473f-acf8-f45e8b101512 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:16.962382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad584f-c7b2-473f-acf8-f45e8b101512 None admin] PATH_INFO: `/v3/users/2c4358ea27094a2aacf8562effae67ee` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:16.963605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad584f-c7b2-473f-acf8-f45e8b101512 None admin] RBAC: Authorizing `identity:get_user(user_id=2c4358ea27094a2aacf8562effae67ee)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:16.965178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad584f-c7b2-473f-acf8-f45e8b101512 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:16.971583 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 134/292] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:16 2022] GET /identity/v3/users/2c4358ea27094a2aacf8562effae67ee => generated 297 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:16.972989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfd1aa-f3a7-4990-99fc-1fcb13e6411b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:16.973290 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfd1aa-f3a7-4990-99fc-1fcb13e6411b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:16.973562 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfd1aa-f3a7-4990-99fc-1fcb13e6411b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:16.975219 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 135/293] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:16.979920 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfd1aa-f3a7-4990-99fc-1fcb13e6411b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:16.988475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:17.004548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:17.034332 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 159/294] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:17.034332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44810-360d-403c-b9c7-cd82cddc40ad None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.034332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44810-360d-403c-b9c7-cd82cddc40ad None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.034332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44810-360d-403c-b9c7-cd82cddc40ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.060647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-80e44810-360d-403c-b9c7-cd82cddc40ad None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:17.061855 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-80e44810-360d-403c-b9c7-cd82cddc40ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:17.119744 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfd1aa-f3a7-4990-99fc-1fcb13e6411b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:17.132819 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 160/295] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:17.133814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e44810-360d-403c-b9c7-cd82cddc40ad None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:17.161677 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299f1fa-fecb-4962-a867-b08a01e336c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:17.162912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299f1fa-fecb-4962-a867-b08a01e336c6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.163337 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299f1fa-fecb-4962-a867-b08a01e336c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.163749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299f1fa-fecb-4962-a867-b08a01e336c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.254434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299f1fa-fecb-4962-a867-b08a01e336c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:17.256092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299f1fa-fecb-4962-a867-b08a01e336c6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:17.258055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:17.263651 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e44810-360d-403c-b9c7-cd82cddc40ad None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:17.281171 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:17.302040 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63ea59-907a-4228-9533-80bf2e5c54c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:17.303240 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63ea59-907a-4228-9533-80bf2e5c54c2 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.303665 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63ea59-907a-4228-9533-80bf2e5c54c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.304088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63ea59-907a-4228-9533-80bf2e5c54c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.314320 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63ea59-907a-4228-9533-80bf2e5c54c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-0e63ea59-907a-4228-9533-80bf2e5c54c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:17.322177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:17.323112 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:17.323903 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 161/296] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:17.324558 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63ea59-907a-4228-9533-80bf2e5c54c2 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:17.372860 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 136/297] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:17.372860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3b4ca-6a27-4e55-81d7-4096550b38ab None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.372860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3b4ca-6a27-4e55-81d7-4096550b38ab None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.372860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3b4ca-6a27-4e55-81d7-4096550b38ab None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.372860 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 137/298] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:17.372860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3b4ca-6a27-4e55-81d7-4096550b38ab None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:17.430103 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63ea59-907a-4228-9533-80bf2e5c54c2 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:17.464297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558ac14-3aae-42c7-b202-5a02341fc3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:17.465111 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558ac14-3aae-42c7-b202-5a02341fc3db None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.465393 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558ac14-3aae-42c7-b202-5a02341fc3db None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.465664 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558ac14-3aae-42c7-b202-5a02341fc3db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.467079 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558ac14-3aae-42c7-b202-5a02341fc3db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:17.468551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558ac14-3aae-42c7-b202-5a02341fc3db None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:17.474682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3b4ca-6a27-4e55-81d7-4096550b38ab None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:17.474823 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 162/299] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:17.474823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558ac14-3aae-42c7-b202-5a02341fc3db None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:17.496996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:17.501073 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:17.511107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3986404-46fa-47cc-94c6-4b228e722218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:17.511808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986404-46fa-47cc-94c6-4b228e722218 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.512092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986404-46fa-47cc-94c6-4b228e722218 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.512365 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986404-46fa-47cc-94c6-4b228e722218 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.513114 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3986404-46fa-47cc-94c6-4b228e722218 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:17.513418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3986404-46fa-47cc-94c6-4b228e722218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:17.515215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3986404-46fa-47cc-94c6-4b228e722218 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:17.524021 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 138/300] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:17.525417 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f175c8-504a-4536-bc56-ff398695b73b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.525772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f175c8-504a-4536-bc56-ff398695b73b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.526111 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f175c8-504a-4536-bc56-ff398695b73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.543694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-09f175c8-504a-4536-bc56-ff398695b73b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:17.545269 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-09f175c8-504a-4536-bc56-ff398695b73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:17.582436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558ac14-3aae-42c7-b202-5a02341fc3db None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:17.615168 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 139/301] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:17.615962 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f175c8-504a-4536-bc56-ff398695b73b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:17.628216 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b42fb-83ba-45e1-81f7-dc9802c4e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:17.629373 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b42fb-83ba-45e1-81f7-dc9802c4e40b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.629792 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b42fb-83ba-45e1-81f7-dc9802c4e40b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.630201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b42fb-83ba-45e1-81f7-dc9802c4e40b None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.631650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b42fb-83ba-45e1-81f7-dc9802c4e40b None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:17.633335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b42fb-83ba-45e1-81f7-dc9802c4e40b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:17.635833 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 163/302] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:17 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:17.636783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b42fb-83ba-45e1-81f7-dc9802c4e40b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:17.731338 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:17.732970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f175c8-504a-4536-bc56-ff398695b73b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:17.739664 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b42fb-83ba-45e1-81f7-dc9802c4e40b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:17.748469 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:17.779313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83405d9a-327c-463d-ab1e-d5b2b109e882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:17.779996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83405d9a-327c-463d-ab1e-d5b2b109e882 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.780282 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83405d9a-327c-463d-ab1e-d5b2b109e882 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.780564 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83405d9a-327c-463d-ab1e-d5b2b109e882 None admin] PATH_INFO: `/v3/groups` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.781102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980d80e-f49e-47f7-94e5-b229b091aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:17.781273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83405d9a-327c-463d-ab1e-d5b2b109e882 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:17.781980 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980d80e-f49e-47f7-94e5-b229b091aa7c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.782293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980d80e-f49e-47f7-94e5-b229b091aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.782567 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980d80e-f49e-47f7-94e5-b229b091aa7c None admin] PATH_INFO: `/v3/users/e0a959135b3542aab1864503c6d6a425` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.782734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83405d9a-327c-463d-ab1e-d5b2b109e882 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:17.783745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980d80e-f49e-47f7-94e5-b229b091aa7c None admin] RBAC: Authorizing `identity:get_user(user_id=e0a959135b3542aab1864503c6d6a425)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:17.785161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980d80e-f49e-47f7-94e5-b229b091aa7c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:17.786995 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 164/303] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:17 2022] GET /identity/v3/users/e0a959135b3542aab1864503c6d6a425 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:17.792845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980d80e-f49e-47f7-94e5-b229b091aa7c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:17.800653 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 140/304] 10.222.0.16 () {60 vars in 1134 bytes} [Tue Jul 5 09:59:17 2022] POST /identity/v3/groups => generated 229 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:59:17.800653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83405d9a-327c-463d-ab1e-d5b2b109e882 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:17.924768 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83405d9a-327c-463d-ab1e-d5b2b109e882 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:17.934989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980d80e-f49e-47f7-94e5-b229b091aa7c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:17.961997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ef737-b6b1-4672-96f2-63e90c810c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:17.962939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ef737-b6b1-4672-96f2-63e90c810c32 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.963220 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ef737-b6b1-4672-96f2-63e90c810c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.963493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ef737-b6b1-4672-96f2-63e90c810c32 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.974789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ef737-b6b1-4672-96f2-63e90c810c32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:17.979698 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bd8b6-ffdd-4bc4-84bd-959df6d51311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:17.980512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bd8b6-ffdd-4bc4-84bd-959df6d51311 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:17.980968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bd8b6-ffdd-4bc4-84bd-959df6d51311 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:17.981347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bd8b6-ffdd-4bc4-84bd-959df6d51311 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:17.983204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7bd8b6-ffdd-4bc4-84bd-959df6d51311 None admin] RBAC: Adding query filter params (role.id=b10cd0627cc84a9c92dba333901e60e1, scope.project.id=36b45cb287dd49428bc291b9834d0b1e, user.id=2c4358ea27094a2aacf8562effae67ee) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c60ef737-b6b1-4672-96f2-63e90c810c32 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:17.983302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:17.984191 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:17.984960 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:17.985558 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7bd8b6-ffdd-4bc4-84bd-959df6d51311 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:17.985632 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 141/305] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:17.985632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ef737-b6b1-4672-96f2-63e90c810c32 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:17.985749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7bd8b6-ffdd-4bc4-84bd-959df6d51311 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:17.993198 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 165/306] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:17 2022] GET /identity/v3/role_assignments?user.id=2c4358ea27094a2aacf8562effae67ee&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:17.994452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bd8b6-ffdd-4bc4-84bd-959df6d51311 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:18.031395 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:18.036893 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:18.112119 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ef737-b6b1-4672-96f2-63e90c810c32 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:18.152655 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bd8b6-ffdd-4bc4-84bd-959df6d51311 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:18.156760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa03b2-3b9f-4406-89e2-d6fd74fcdea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:18.157661 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa03b2-3b9f-4406-89e2-d6fd74fcdea1 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:18.157942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa03b2-3b9f-4406-89e2-d6fd74fcdea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:18.158273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa03b2-3b9f-4406-89e2-d6fd74fcdea1 None admin] PATH_INFO: `/v3/system/users/e0a959135b3542aab1864503c6d6a425/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:18.159885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa03b2-3b9f-4406-89e2-d6fd74fcdea1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e0a959135b3542aab1864503c6d6a425, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:18.161304 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa03b2-3b9f-4406-89e2-d6fd74fcdea1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:18.176958 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 142/307] 10.222.0.16 () {58 vars in 1334 bytes} [Tue Jul 5 09:59:17 2022] PUT /identity/v3/system/users/e0a959135b3542aab1864503c6d6a425/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:18.202937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48c167-7fb1-45de-a4b9-0748855e13e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:18.203892 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48c167-7fb1-45de-a4b9-0748855e13e9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:18.204357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48c167-7fb1-45de-a4b9-0748855e13e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:18.204777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48c167-7fb1-45de-a4b9-0748855e13e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:18.205710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48c167-7fb1-45de-a4b9-0748855e13e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:18.206280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48c167-7fb1-45de-a4b9-0748855e13e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:18.208175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48c167-7fb1-45de-a4b9-0748855e13e9 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:18.219126 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 166/308] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:18.226348 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa03b2-3b9f-4406-89e2-d6fd74fcdea1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:18.339001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:18.343386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa03b2-3b9f-4406-89e2-d6fd74fcdea1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:18.380045 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d857e-546c-435e-bd91-0ba827def304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:18.381262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d857e-546c-435e-bd91-0ba827def304 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:18.381695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d857e-546c-435e-bd91-0ba827def304 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:18.382113 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d857e-546c-435e-bd91-0ba827def304 None admin] PATH_INFO: `/v3/users/aecc43465e334fa49e2dca892a616405` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:18.396856 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d857e-546c-435e-bd91-0ba827def304 None admin] RBAC: Authorizing `identity:get_user(user_id=aecc43465e334fa49e2dca892a616405)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:18.398595 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d857e-546c-435e-bd91-0ba827def304 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:18.400820 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 143/309] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:18 2022] GET /identity/v3/users/aecc43465e334fa49e2dca892a616405 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:18.406606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48c167-7fb1-45de-a4b9-0748855e13e9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:18.500664 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:18.514430 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48c167-7fb1-45de-a4b9-0748855e13e9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:18.549085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc440f04-8af5-4aca-ba08-49dd3cbf1745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:18.549995 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc440f04-8af5-4aca-ba08-49dd3cbf1745 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:18.550462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc440f04-8af5-4aca-ba08-49dd3cbf1745 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:18.550927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc440f04-8af5-4aca-ba08-49dd3cbf1745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:18.551938 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc440f04-8af5-4aca-ba08-49dd3cbf1745 None admin] RBAC: Adding query filter params (role.id=8646f386c6704cb190f12a4b4e1a2245, user.id=aecc43465e334fa49e2dca892a616405, scope.system=all) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:18.552482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc440f04-8af5-4aca-ba08-49dd3cbf1745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:18.554087 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc440f04-8af5-4aca-ba08-49dd3cbf1745 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:18.559894 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 167/310] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Jul 5 09:59:18 2022] GET /identity/v3/role_assignments?user.id=aecc43465e334fa49e2dca892a616405&scope.system=all&role.id=8646f386c6704cb190f12a4b4e1a2245 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:18.712058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa416c0-da94-423a-a271-044de440e2a8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:18.712346 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa416c0-da94-423a-a271-044de440e2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:18.712627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa416c0-da94-423a-a271-044de440e2a8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:18.714015 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 144/311] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:18.719480 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7dd1-3f2b-4b72-b06a-14810d22102a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:18.719946 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7dd1-3f2b-4b72-b06a-14810d22102a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:18.720032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7dd1-3f2b-4b72-b06a-14810d22102a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:18.738040 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-fb6e7dd1-3f2b-4b72-b06a-14810d22102a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:18.739295 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-fb6e7dd1-3f2b-4b72-b06a-14810d22102a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:18.814129 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 168/312] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:18.838091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa416c0-da94-423a-a271-044de440e2a8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:18.944006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa416c0-da94-423a-a271-044de440e2a8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:18.979029 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea331e-5c1a-4f2a-941c-130d26ab7461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:18.979909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea331e-5c1a-4f2a-941c-130d26ab7461 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:18.980195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea331e-5c1a-4f2a-941c-130d26ab7461 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:18.980527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea331e-5c1a-4f2a-941c-130d26ab7461 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:18.991298 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea331e-5c1a-4f2a-941c-130d26ab7461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-44ea331e-5c1a-4f2a-941c-130d26ab7461 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:18.999284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:19.000150 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:19.000954 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 145/313] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:19.006658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e7dd1-3f2b-4b72-b06a-14810d22102a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:19.089992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:19.110937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e7dd1-3f2b-4b72-b06a-14810d22102a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:19.147115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197f4a7-10d4-4573-8cdf-bf0c76a25037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:19.148009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197f4a7-10d4-4573-8cdf-bf0c76a25037 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:19.148293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197f4a7-10d4-4573-8cdf-bf0c76a25037 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:19.148568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197f4a7-10d4-4573-8cdf-bf0c76a25037 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:19.149317 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197f4a7-10d4-4573-8cdf-bf0c76a25037 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:19.149623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197f4a7-10d4-4573-8cdf-bf0c76a25037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:19.151110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197f4a7-10d4-4573-8cdf-bf0c76a25037 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:19.159698 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 169/314] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:19.164967 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea331e-5c1a-4f2a-941c-130d26ab7461 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:19.183971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:19.267394 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea331e-5c1a-4f2a-941c-130d26ab7461 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:19.300472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0140f-3544-4e56-96ff-2022dbe4080d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:19.301359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0140f-3544-4e56-96ff-2022dbe4080d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:19.301643 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0140f-3544-4e56-96ff-2022dbe4080d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:19.301921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0140f-3544-4e56-96ff-2022dbe4080d None admin] PATH_INFO: `/v3/users/dad4a61888834a5ea6c92cdc9d448a0e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:19.303243 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0140f-3544-4e56-96ff-2022dbe4080d None admin] RBAC: Authorizing `identity:get_user(user_id=dad4a61888834a5ea6c92cdc9d448a0e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:19.304742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0140f-3544-4e56-96ff-2022dbe4080d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:19.306728 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 146/315] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:19 2022] GET /identity/v3/users/dad4a61888834a5ea6c92cdc9d448a0e => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:19.315329 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197f4a7-10d4-4573-8cdf-bf0c76a25037 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:19.421433 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197f4a7-10d4-4573-8cdf-bf0c76a25037 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:19.455012 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e147a-9125-4d03-8bc3-c9979c8a29e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:19.455672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e147a-9125-4d03-8bc3-c9979c8a29e3 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:19.455954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e147a-9125-4d03-8bc3-c9979c8a29e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:19.456229 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e147a-9125-4d03-8bc3-c9979c8a29e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:19.471883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:19.527025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e147a-9125-4d03-8bc3-c9979c8a29e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:19.528635 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e147a-9125-4d03-8bc3-c9979c8a29e3 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:19.627337 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 170/316] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:19.632809 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0140f-3544-4e56-96ff-2022dbe4080d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:19.723482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:19.741896 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0140f-3544-4e56-96ff-2022dbe4080d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:19.785252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca879186-0558-4e75-bb37-3fc8219d285d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:19.786116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca879186-0558-4e75-bb37-3fc8219d285d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:19.786438 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca879186-0558-4e75-bb37-3fc8219d285d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:19.786818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca879186-0558-4e75-bb37-3fc8219d285d None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:19.788010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca879186-0558-4e75-bb37-3fc8219d285d None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:19.789462 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca879186-0558-4e75-bb37-3fc8219d285d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:19.791510 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 147/317] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:19 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:19.799354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e147a-9125-4d03-8bc3-c9979c8a29e3 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:19.844681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:19.900579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e147a-9125-4d03-8bc3-c9979c8a29e3 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:19.934197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5797e-f674-49f7-85a6-60d7d26c2a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:19.935116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5797e-f674-49f7-85a6-60d7d26c2a86 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:19.935552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5797e-f674-49f7-85a6-60d7d26c2a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:19.935990 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5797e-f674-49f7-85a6-60d7d26c2a86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:19.936941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5797e-f674-49f7-85a6-60d7d26c2a86 None admin] RBAC: Adding query filter params (role.id=8646f386c6704cb190f12a4b4e1a2245, scope.project.id=36b45cb287dd49428bc291b9834d0b1e, user.id=dad4a61888834a5ea6c92cdc9d448a0e) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:19.937384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5797e-f674-49f7-85a6-60d7d26c2a86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:19.939005 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5797e-f674-49f7-85a6-60d7d26c2a86 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:19.945538 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 171/318] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:19 2022] GET /identity/v3/role_assignments?user.id=dad4a61888834a5ea6c92cdc9d448a0e&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=8646f386c6704cb190f12a4b4e1a2245 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:20.120872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59d173-2c45-4b57-8bf8-94bb2634189b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:20.121240 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59d173-2c45-4b57-8bf8-94bb2634189b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:20.121434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59d173-2c45-4b57-8bf8-94bb2634189b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:20.123209 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 148/319] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:20.128988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6a2d7-7c64-4b84-8d6b-b4508314d694 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:20.129618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6a2d7-7c64-4b84-8d6b-b4508314d694 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:20.130083 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6a2d7-7c64-4b84-8d6b-b4508314d694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:20.148493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-0cb6a2d7-7c64-4b84-8d6b-b4508314d694 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:20.149985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0cb6a2d7-7c64-4b84-8d6b-b4508314d694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:20.220939 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 172/320] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:20.242553 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59d173-2c45-4b57-8bf8-94bb2634189b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:20.349500 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59d173-2c45-4b57-8bf8-94bb2634189b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:20.383367 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aba139-ae16-4e31-9c7c-f3e459604bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:20.384090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aba139-ae16-4e31-9c7c-f3e459604bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:20.384374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aba139-ae16-4e31-9c7c-f3e459604bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:20.384644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aba139-ae16-4e31-9c7c-f3e459604bd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:20.385821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aba139-ae16-4e31-9c7c-f3e459604bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:20.387382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aba139-ae16-4e31-9c7c-f3e459604bd4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:20.389163 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 149/321] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:20.394749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6a2d7-7c64-4b84-8d6b-b4508314d694 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:20.431161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614b7c-08ad-4570-96ac-c65371864082 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:20.431735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614b7c-08ad-4570-96ac-c65371864082 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:20.432012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614b7c-08ad-4570-96ac-c65371864082 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:20.433558 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 150/322] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:20.437848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:20.438796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74497f39-ef89-4418-8c06-111f6df46e6c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:20.439089 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74497f39-ef89-4418-8c06-111f6df46e6c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:20.439361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74497f39-ef89-4418-8c06-111f6df46e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:20.457442 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-74497f39-ef89-4418-8c06-111f6df46e6c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:20.459098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-74497f39-ef89-4418-8c06-111f6df46e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:20.473565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:20.504698 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6a2d7-7c64-4b84-8d6b-b4508314d694 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:20.538175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc124a7-0037-4a01-b881-8aef0bd5372a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:20.538394 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 151/323] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:20.538874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc124a7-0037-4a01-b881-8aef0bd5372a None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:20.539167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc124a7-0037-4a01-b881-8aef0bd5372a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:20.539444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc124a7-0037-4a01-b881-8aef0bd5372a None admin] PATH_INFO: `/v3/groups` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:20.540140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc124a7-0037-4a01-b881-8aef0bd5372a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:20.541609 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc124a7-0037-4a01-b881-8aef0bd5372a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:20.556285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74497f39-ef89-4418-8c06-111f6df46e6c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:20.558819 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 173/324] 10.222.0.16 () {60 vars in 1134 bytes} [Tue Jul 5 09:59:20 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:59:20.666814 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74497f39-ef89-4418-8c06-111f6df46e6c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:20.667765 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2030f9d-2cf4-4e57-8960-a7d5751c55da None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:20.668051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2030f9d-2cf4-4e57-8960-a7d5751c55da None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:20.668614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2030f9d-2cf4-4e57-8960-a7d5751c55da None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:20.671547 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 174/325] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:20.677504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a48c3-88ac-4eed-bdad-33335f3706d3 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:20.677879 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a48c3-88ac-4eed-bdad-33335f3706d3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:20.678060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a48c3-88ac-4eed-bdad-33335f3706d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:20.697399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-eb9a48c3-88ac-4eed-bdad-33335f3706d3 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:20.698805 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-eb9a48c3-88ac-4eed-bdad-33335f3706d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:20.705438 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6cf3f-3f2d-4e89-8c89-127f53778127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:20.706290 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6cf3f-3f2d-4e89-8c89-127f53778127 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:20.706574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6cf3f-3f2d-4e89-8c89-127f53778127 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:20.706852 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6cf3f-3f2d-4e89-8c89-127f53778127 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:20.720501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6cf3f-3f2d-4e89-8c89-127f53778127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-34c6cf3f-3f2d-4e89-8c89-127f53778127 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:20.728280 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:20.729140 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:20.730159 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 152/326] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:20.735841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6cf3f-3f2d-4e89-8c89-127f53778127 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:20.748566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:20.785237 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 175/327] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:20.807137 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a48c3-88ac-4eed-bdad-33335f3706d3 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:20.839845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:20.847303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6cf3f-3f2d-4e89-8c89-127f53778127 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:20.889440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82f0e1-fcf2-491b-b6e2-79e3736423d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:20.890130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82f0e1-fcf2-491b-b6e2-79e3736423d3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:20.890482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82f0e1-fcf2-491b-b6e2-79e3736423d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:20.890764 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82f0e1-fcf2-491b-b6e2-79e3736423d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:20.892798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82f0e1-fcf2-491b-b6e2-79e3736423d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:20.892798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82f0e1-fcf2-491b-b6e2-79e3736423d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:20.894183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82f0e1-fcf2-491b-b6e2-79e3736423d3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:20.910966 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 153/328] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:20.918687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82f0e1-fcf2-491b-b6e2-79e3736423d3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:20.943705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a48c3-88ac-4eed-bdad-33335f3706d3 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:20.984583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec160c-389d-47b4-ae97-9572f2daa8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:20.985419 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec160c-389d-47b4-ae97-9572f2daa8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:20.985706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec160c-389d-47b4-ae97-9572f2daa8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:20.985977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec160c-389d-47b4-ae97-9572f2daa8b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:20.999180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec160c-389d-47b4-ae97-9572f2daa8b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-12ec160c-389d-47b4-ae97-9572f2daa8b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:21.007195 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:21.008124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:21.009393 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 176/329] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:21.015088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:21.016345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec160c-389d-47b4-ae97-9572f2daa8b4 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:21.055870 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82f0e1-fcf2-491b-b6e2-79e3736423d3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:21.103662 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3be86-a4a8-4c73-bfb4-4906e3edbdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:21.104567 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3be86-a4a8-4c73-bfb4-4906e3edbdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:21.104872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3be86-a4a8-4c73-bfb4-4906e3edbdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:21.105398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3be86-a4a8-4c73-bfb4-4906e3edbdb3 None admin] PATH_INFO: `/v3/users/e0a959135b3542aab1864503c6d6a425` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:21.106716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3be86-a4a8-4c73-bfb4-4906e3edbdb3 None admin] RBAC: Authorizing `identity:get_user(user_id=e0a959135b3542aab1864503c6d6a425)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:21.109412 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3be86-a4a8-4c73-bfb4-4906e3edbdb3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:21.111578 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 154/330] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:20 2022] GET /identity/v3/users/e0a959135b3542aab1864503c6d6a425 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:21.118432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3be86-a4a8-4c73-bfb4-4906e3edbdb3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:21.130978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:21.148438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec160c-389d-47b4-ae97-9572f2daa8b4 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:21.198378 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807928b-6d91-48e1-8a45-717c5b6c258d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:21.199152 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807928b-6d91-48e1-8a45-717c5b6c258d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:21.199430 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807928b-6d91-48e1-8a45-717c5b6c258d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:21.200017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807928b-6d91-48e1-8a45-717c5b6c258d None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:21.201049 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807928b-6d91-48e1-8a45-717c5b6c258d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:21.201356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807928b-6d91-48e1-8a45-717c5b6c258d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:21.203249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807928b-6d91-48e1-8a45-717c5b6c258d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:21.215783 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 177/331] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:21.224643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807928b-6d91-48e1-8a45-717c5b6c258d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:21.260040 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3be86-a4a8-4c73-bfb4-4906e3edbdb3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:21.305078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480e8d8-f2e0-409d-8beb-eb2f9a75e2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:21.306026 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480e8d8-f2e0-409d-8beb-eb2f9a75e2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:21.306342 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480e8d8-f2e0-409d-8beb-eb2f9a75e2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:21.306868 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480e8d8-f2e0-409d-8beb-eb2f9a75e2e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:21.308133 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480e8d8-f2e0-409d-8beb-eb2f9a75e2e4 None admin] RBAC: Adding query filter params (role.id=b10cd0627cc84a9c92dba333901e60e1, user.id=e0a959135b3542aab1864503c6d6a425, scope.system=all) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:21.308708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480e8d8-f2e0-409d-8beb-eb2f9a75e2e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:21.310449 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480e8d8-f2e0-409d-8beb-eb2f9a75e2e4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:21.317664 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:21.327765 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 155/332] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Jul 5 09:59:21 2022] GET /identity/v3/role_assignments?user.id=e0a959135b3542aab1864503c6d6a425&scope.system=all&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:21.363705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807928b-6d91-48e1-8a45-717c5b6c258d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:21.402167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625f17d-257d-4688-987d-8c5f1affe60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:21.403089 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625f17d-257d-4688-987d-8c5f1affe60f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:21.403371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625f17d-257d-4688-987d-8c5f1affe60f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:21.403643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625f17d-257d-4688-987d-8c5f1affe60f None admin] PATH_INFO: `/v3/users/aecc43465e334fa49e2dca892a616405` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:21.404863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625f17d-257d-4688-987d-8c5f1affe60f None admin] RBAC: Authorizing `identity:get_user(user_id=aecc43465e334fa49e2dca892a616405)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:21.406382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625f17d-257d-4688-987d-8c5f1affe60f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:21.408427 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 178/333] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:21 2022] GET /identity/v3/users/aecc43465e334fa49e2dca892a616405 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:21.414318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480e8d8-f2e0-409d-8beb-eb2f9a75e2e4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:21.544891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480e8d8-f2e0-409d-8beb-eb2f9a75e2e4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:21.585780 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d93aa-3115-44f6-bb06-e3a05f213f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:21.586683 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d93aa-3115-44f6-bb06-e3a05f213f23 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:21.586945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d93aa-3115-44f6-bb06-e3a05f213f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:21.587227 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d93aa-3115-44f6-bb06-e3a05f213f23 None admin] PATH_INFO: `/v3/system/users/aecc43465e334fa49e2dca892a616405/roles/8646f386c6704cb190f12a4b4e1a2245` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:21.588731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d93aa-3115-44f6-bb06-e3a05f213f23 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=aecc43465e334fa49e2dca892a616405, role_id=8646f386c6704cb190f12a4b4e1a2245)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:21.590131 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d93aa-3115-44f6-bb06-e3a05f213f23 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:21.600323 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 156/334] 10.222.0.16 () {58 vars in 1334 bytes} [Tue Jul 5 09:59:21 2022] PUT /identity/v3/system/users/aecc43465e334fa49e2dca892a616405/roles/8646f386c6704cb190f12a4b4e1a2245 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:23.135704 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a16ccb-344a-4b5c-bae6-60da9de7ceda None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:23.136206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a16ccb-344a-4b5c-bae6-60da9de7ceda None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:23.136206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a16ccb-344a-4b5c-bae6-60da9de7ceda None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:23.137533 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 179/335] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:23.147673 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055f177-9b56-467b-9972-cd794fcce886 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:23.148554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055f177-9b56-467b-9972-cd794fcce886 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:23.149032 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055f177-9b56-467b-9972-cd794fcce886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:23.171288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b055f177-9b56-467b-9972-cd794fcce886 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:23.172843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b055f177-9b56-467b-9972-cd794fcce886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:23.242050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:23.258948 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 157/336] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:23.297279 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a16ccb-344a-4b5c-bae6-60da9de7ceda None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:23.316282 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ed8f5-9091-4c58-95da-4453df041772 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:23.316918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ed8f5-9091-4c58-95da-4453df041772 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:23.317401 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ed8f5-9091-4c58-95da-4453df041772 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:23.319378 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 158/337] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:23.324408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc096-4255-4b63-9e01-36305347132a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:23.324935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc096-4255-4b63-9e01-36305347132a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:23.325430 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc096-4255-4b63-9e01-36305347132a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:23.328291 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:23.351288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d02fc096-4255-4b63-9e01-36305347132a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:23.352871 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d02fc096-4255-4b63-9e01-36305347132a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:23.457590 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 159/338] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:23.471881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a16ccb-344a-4b5c-bae6-60da9de7ceda None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:23.501178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fc096-4255-4b63-9e01-36305347132a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:23.517644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13008fcc-2bf2-40ed-8bd5-b8670e35e037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:23.518545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13008fcc-2bf2-40ed-8bd5-b8670e35e037 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:23.518826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13008fcc-2bf2-40ed-8bd5-b8670e35e037 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:23.519096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13008fcc-2bf2-40ed-8bd5-b8670e35e037 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:23.537408 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13008fcc-2bf2-40ed-8bd5-b8670e35e037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-13008fcc-2bf2-40ed-8bd5-b8670e35e037 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:23.550314 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:23.551238 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:23.552048 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 180/339] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:23.552783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13008fcc-2bf2-40ed-8bd5-b8670e35e037 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:23.647129 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:23.649331 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fc096-4255-4b63-9e01-36305347132a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:23.671284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13008fcc-2bf2-40ed-8bd5-b8670e35e037 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:23.706998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:23.716220 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf655d79-b202-4b04-a244-b4c39658b225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:23.716848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf655d79-b202-4b04-a244-b4c39658b225 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:23.717127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf655d79-b202-4b04-a244-b4c39658b225 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:23.717403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf655d79-b202-4b04-a244-b4c39658b225 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:23.718157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf655d79-b202-4b04-a244-b4c39658b225 None admin] RBAC: Adding query filter params (name=member) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:23.718500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf655d79-b202-4b04-a244-b4c39658b225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:23.720025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf655d79-b202-4b04-a244-b4c39658b225 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:23.732188 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 181/340] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:23.737752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf655d79-b202-4b04-a244-b4c39658b225 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:23.743514 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a85f93-a801-4f88-897d-a172711ea2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:23.744429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a85f93-a801-4f88-897d-a172711ea2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:23.744717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a85f93-a801-4f88-897d-a172711ea2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:23.744994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a85f93-a801-4f88-897d-a172711ea2d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:23.771516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a85f93-a801-4f88-897d-a172711ea2d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a5a85f93-a801-4f88-897d-a172711ea2d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:23.788771 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:23.789734 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:23.790544 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 160/341] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:59:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:23.793143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3fdcd-9d7f-406b-97f8-c494e19f335b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:23.793143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3fdcd-9d7f-406b-97f8-c494e19f335b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:23.793143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3fdcd-9d7f-406b-97f8-c494e19f335b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:23.793992 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 161/342] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:23.798626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3fdcd-9d7f-406b-97f8-c494e19f335b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:23.846578 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf655d79-b202-4b04-a244-b4c39658b225 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:23.880232 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8aa6a-f66e-4f64-a36d-5ff89a3dec3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:23.880954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8aa6a-f66e-4f64-a36d-5ff89a3dec3a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:23.881237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8aa6a-f66e-4f64-a36d-5ff89a3dec3a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:23.881509 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8aa6a-f66e-4f64-a36d-5ff89a3dec3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:23.892215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:23.949877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8aa6a-f66e-4f64-a36d-5ff89a3dec3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:23.951363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8aa6a-f66e-4f64-a36d-5ff89a3dec3a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:23.976096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3fdcd-9d7f-406b-97f8-c494e19f335b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:24.028463 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6c18d-cd3b-483d-b57b-42a68ce41278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:24.029159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6c18d-cd3b-483d-b57b-42a68ce41278 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:24.029446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6c18d-cd3b-483d-b57b-42a68ce41278 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:24.029724 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6c18d-cd3b-483d-b57b-42a68ce41278 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:24.030609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6c18d-cd3b-483d-b57b-42a68ce41278 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:24.030952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6c18d-cd3b-483d-b57b-42a68ce41278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:24.032492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6c18d-cd3b-483d-b57b-42a68ce41278 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:24.049826 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 162/343] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:24.050608 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 182/344] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:24.051170 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec1671-325a-40dd-9c4d-2e694598b9e2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:24.051465 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec1671-325a-40dd-9c4d-2e694598b9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:24.051742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec1671-325a-40dd-9c4d-2e694598b9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:24.055173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8aa6a-f66e-4f64-a36d-5ff89a3dec3a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:24.079856 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-76ec1671-325a-40dd-9c4d-2e694598b9e2 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:24.081210 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-76ec1671-325a-40dd-9c4d-2e694598b9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:24.107463 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:24.142548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:24.171644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8aa6a-f66e-4f64-a36d-5ff89a3dec3a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:24.213119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693a86f-e99a-4dc6-9d47-a965976429a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:24.214071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693a86f-e99a-4dc6-9d47-a965976429a3 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:24.214547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693a86f-e99a-4dc6-9d47-a965976429a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:24.215333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693a86f-e99a-4dc6-9d47-a965976429a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:24.256007 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 163/345] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:24.256964 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec1671-325a-40dd-9c4d-2e694598b9e2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:24.286196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693a86f-e99a-4dc6-9d47-a965976429a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:24.288132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693a86f-e99a-4dc6-9d47-a965976429a3 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:24.330575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:24.378833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec1671-325a-40dd-9c4d-2e694598b9e2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:24.416550 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77a47c-ce51-4bbd-b7a1-8bead9e8d4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:24.417427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77a47c-ce51-4bbd-b7a1-8bead9e8d4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:24.417711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77a47c-ce51-4bbd-b7a1-8bead9e8d4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:24.417989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77a47c-ce51-4bbd-b7a1-8bead9e8d4f4 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:24.419217 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77a47c-ce51-4bbd-b7a1-8bead9e8d4f4 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:24.419495 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 183/346] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:24.421349 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693a86f-e99a-4dc6-9d47-a965976429a3 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:24.425138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77a47c-ce51-4bbd-b7a1-8bead9e8d4f4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:24.426968 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 164/347] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:24 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:24.429055 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77a47c-ce51-4bbd-b7a1-8bead9e8d4f4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:24.480099 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:24.588579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693a86f-e99a-4dc6-9d47-a965976429a3 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:24.590704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77a47c-ce51-4bbd-b7a1-8bead9e8d4f4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:24.636970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3768bb-4046-4b50-92e1-04e2d234b188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:24.637783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3768bb-4046-4b50-92e1-04e2d234b188 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:24.638065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3768bb-4046-4b50-92e1-04e2d234b188 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:24.638446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3768bb-4046-4b50-92e1-04e2d234b188 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:24.649686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3768bb-4046-4b50-92e1-04e2d234b188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:24.658573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-1a3768bb-4046-4b50-92e1-04e2d234b188 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:24.661914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:24.662989 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:24.663806 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 184/348] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:24.664464 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3768bb-4046-4b50-92e1-04e2d234b188 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:24.722945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de72013-1b88-48a6-b970-91a7e6cfb1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:24.723912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de72013-1b88-48a6-b970-91a7e6cfb1fb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:24.724245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de72013-1b88-48a6-b970-91a7e6cfb1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:24.724525 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de72013-1b88-48a6-b970-91a7e6cfb1fb None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:24.728005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de72013-1b88-48a6-b970-91a7e6cfb1fb None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:24.729529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de72013-1b88-48a6-b970-91a7e6cfb1fb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:24.731665 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 165/349] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:24 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:24.732440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de72013-1b88-48a6-b970-91a7e6cfb1fb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:24.793271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3768bb-4046-4b50-92e1-04e2d234b188 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:24.829647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4849be5-cc03-4bcb-b977-b89fcf7a95f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:24.830563 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4849be5-cc03-4bcb-b977-b89fcf7a95f5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:24.830848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4849be5-cc03-4bcb-b977-b89fcf7a95f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:24.831122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4849be5-cc03-4bcb-b977-b89fcf7a95f5 None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:24.837817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4849be5-cc03-4bcb-b977-b89fcf7a95f5 None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:24.839386 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4849be5-cc03-4bcb-b977-b89fcf7a95f5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:24.841860 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 185/350] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:24 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:24.842748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4849be5-cc03-4bcb-b977-b89fcf7a95f5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:24.885740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de72013-1b88-48a6-b970-91a7e6cfb1fb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:24.897536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:24.906064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:24.926818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6419a-cf80-4776-9f97-96777aaee34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:24.927962 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6419a-cf80-4776-9f97-96777aaee34c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:24.928530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6419a-cf80-4776-9f97-96777aaee34c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:24.929006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6419a-cf80-4776-9f97-96777aaee34c None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:24.930004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6419a-cf80-4776-9f97-96777aaee34c None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:24.930732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6419a-cf80-4776-9f97-96777aaee34c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:24.932429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6419a-cf80-4776-9f97-96777aaee34c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:24.942193 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 166/351] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:24.944112 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeeb88a-d7fb-4602-8f5e-ab91463f99c1 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:24.944627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeeb88a-d7fb-4602-8f5e-ab91463f99c1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:24.945120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeeb88a-d7fb-4602-8f5e-ab91463f99c1 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:24.946646 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 167/352] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:24.946752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4849be5-cc03-4bcb-b977-b89fcf7a95f5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:24.947682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeeb88a-d7fb-4602-8f5e-ab91463f99c1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:24.983089 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdcd62-ad74-492b-b402-659efb2fd61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:24.983967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdcd62-ad74-492b-b402-659efb2fd61a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:24.984271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdcd62-ad74-492b-b402-659efb2fd61a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:24.984534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdcd62-ad74-492b-b402-659efb2fd61a None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:24.986815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdcd62-ad74-492b-b402-659efb2fd61a None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:24.988611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdcd62-ad74-492b-b402-659efb2fd61a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:24.989946 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 186/353] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:24 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:24.994459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdcd62-ad74-492b-b402-659efb2fd61a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:25.070664 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeeb88a-d7fb-4602-8f5e-ab91463f99c1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:25.121102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ccffc-a29f-48a2-8ee0-6b929dfecf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:25.121921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ccffc-a29f-48a2-8ee0-6b929dfecf56 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.122600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ccffc-a29f-48a2-8ee0-6b929dfecf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.122896 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ccffc-a29f-48a2-8ee0-6b929dfecf56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.123996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ccffc-a29f-48a2-8ee0-6b929dfecf56 None admin] RBAC: Adding query filter params (group.id=d1cb6cdec2244e5bb303a3b659d83293, role.id=b10cd0627cc84a9c92dba333901e60e1, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:25.124312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ccffc-a29f-48a2-8ee0-6b929dfecf56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:25.125165 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdcd62-ad74-492b-b402-659efb2fd61a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:25.125810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ccffc-a29f-48a2-8ee0-6b929dfecf56 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:25.136423 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 168/354] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jul 5 09:59:24 2022] GET /identity/v3/role_assignments?group.id=d1cb6cdec2244e5bb303a3b659d83293&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:25.136423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e383ce-475e-4f0c-8417-2ab12d8ae1c0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.136423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e383ce-475e-4f0c-8417-2ab12d8ae1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.136423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e383ce-475e-4f0c-8417-2ab12d8ae1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.158427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b9e383ce-475e-4f0c-8417-2ab12d8ae1c0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:25.159514 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:25.160825 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b9e383ce-475e-4f0c-8417-2ab12d8ae1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:25.167356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be2fce-31c1-4c9a-879c-c3bea7783a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:25.168179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be2fce-31c1-4c9a-879c-c3bea7783a29 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.168529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be2fce-31c1-4c9a-879c-c3bea7783a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.168915 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be2fce-31c1-4c9a-879c-c3bea7783a29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.202846 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:25.253414 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be2fce-31c1-4c9a-879c-c3bea7783a29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:25.254886 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be2fce-31c1-4c9a-879c-c3bea7783a29 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:25.255081 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 169/355] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:25.256642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b4fc2-734d-4af9-a1dd-c87bda5fcf4a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.257142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b4fc2-734d-4af9-a1dd-c87bda5fcf4a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.257304 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b4fc2-734d-4af9-a1dd-c87bda5fcf4a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.259119 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 170/356] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:25.260942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b4fc2-734d-4af9-a1dd-c87bda5fcf4a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:25.399880 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 187/357] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:25.401198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffb276-b1eb-438a-a4dc-e956edf08f47 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.401499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffb276-b1eb-438a-a4dc-e956edf08f47 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.401788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffb276-b1eb-438a-a4dc-e956edf08f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.402034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b4fc2-734d-4af9-a1dd-c87bda5fcf4a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:25.419898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-5dffb276-b1eb-438a-a4dc-e956edf08f47 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:25.421120 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5dffb276-b1eb-438a-a4dc-e956edf08f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:25.440742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09481523-bd2e-4bca-8a08-2c819675a8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:25.441713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09481523-bd2e-4bca-8a08-2c819675a8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.441999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09481523-bd2e-4bca-8a08-2c819675a8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.442299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09481523-bd2e-4bca-8a08-2c819675a8e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.443198 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09481523-bd2e-4bca-8a08-2c819675a8e5 None admin] RBAC: Adding query filter params (group.id=d1cb6cdec2244e5bb303a3b659d83293, role.id=70d38d0a968a4e9585c5873d24cfbbb1, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:25.443512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09481523-bd2e-4bca-8a08-2c819675a8e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:25.445487 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09481523-bd2e-4bca-8a08-2c819675a8e5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:25.452929 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 171/358] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jul 5 09:59:25 2022] GET /identity/v3/role_assignments?group.id=d1cb6cdec2244e5bb303a3b659d83293&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=70d38d0a968a4e9585c5873d24cfbbb1 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:25.453596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09481523-bd2e-4bca-8a08-2c819675a8e5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:25.455126 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:25.498505 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:25.517445 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 188/359] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:25.523519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a63c0-a887-46af-87cd-f794465469d0 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.523519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a63c0-a887-46af-87cd-f794465469d0 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.523815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a63c0-a887-46af-87cd-f794465469d0 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.531121 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 189/360] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:25 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:25.532392 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a63c0-a887-46af-87cd-f794465469d0 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:25.596836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09481523-bd2e-4bca-8a08-2c819675a8e5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:25.644392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae0109-5537-4bd0-a2b5-349d2bc44966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:25.645838 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae0109-5537-4bd0-a2b5-349d2bc44966 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.645838 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae0109-5537-4bd0-a2b5-349d2bc44966 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.646125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae0109-5537-4bd0-a2b5-349d2bc44966 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.657616 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae0109-5537-4bd0-a2b5-349d2bc44966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-3eae0109-5537-4bd0-a2b5-349d2bc44966 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:25.667664 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:25.668561 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:25.669394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:25.670008 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 172/361] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:59:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:25.671634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba41b8-2ef2-422e-a179-3f3ccf46b59a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.671634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba41b8-2ef2-422e-a179-3f3ccf46b59a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.671634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba41b8-2ef2-422e-a179-3f3ccf46b59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.690476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-89ba41b8-2ef2-422e-a179-3f3ccf46b59a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:25.692301 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-89ba41b8-2ef2-422e-a179-3f3ccf46b59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:25.699670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a63c0-a887-46af-87cd-f794465469d0 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:25.746131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4e6e-5c04-4f97-8f6a-d372744ed57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:25.747264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4e6e-5c04-4f97-8f6a-d372744ed57d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.747555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4e6e-5c04-4f97-8f6a-d372744ed57d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.747833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4e6e-5c04-4f97-8f6a-d372744ed57d None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.749196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4e6e-5c04-4f97-8f6a-d372744ed57d None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:25.751009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4e6e-5c04-4f97-8f6a-d372744ed57d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:25.753258 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 190/362] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:25 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:25.754071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4e6e-5c04-4f97-8f6a-d372744ed57d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:25.794566 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 173/363] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:25.795955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba41b8-2ef2-422e-a179-3f3ccf46b59a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:25.798316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:25.830006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:25.876897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4e6e-5c04-4f97-8f6a-d372744ed57d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:25.911615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba41b8-2ef2-422e-a179-3f3ccf46b59a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:25.914852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606165ee-8ea2-4607-bd2d-9438b6284224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:25.915679 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606165ee-8ea2-4607-bd2d-9438b6284224 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.915961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606165ee-8ea2-4607-bd2d-9438b6284224 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.916235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606165ee-8ea2-4607-bd2d-9438b6284224 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.927868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606165ee-8ea2-4607-bd2d-9438b6284224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-606165ee-8ea2-4607-bd2d-9438b6284224 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:25.935728 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:25.936643 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:25.937435 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 191/364] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:25.938099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606165ee-8ea2-4607-bd2d-9438b6284224 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:25.948675 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a7d25-91a0-4b7d-9878-abf64d57dff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:25.949369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a7d25-91a0-4b7d-9878-abf64d57dff7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:25.949654 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a7d25-91a0-4b7d-9878-abf64d57dff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:25.949928 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a7d25-91a0-4b7d-9878-abf64d57dff7 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:25.950745 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a7d25-91a0-4b7d-9878-abf64d57dff7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:25.951055 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a7d25-91a0-4b7d-9878-abf64d57dff7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:25.952452 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a7d25-91a0-4b7d-9878-abf64d57dff7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:25.963302 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 174/365] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:25.964246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a7d25-91a0-4b7d-9878-abf64d57dff7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:26.038106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:26.048571 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606165ee-8ea2-4607-bd2d-9438b6284224 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:26.054597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:26.070738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a7d25-91a0-4b7d-9878-abf64d57dff7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:26.084574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a400d6-930c-46d0-82d3-52d40c5f6c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:26.085460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a400d6-930c-46d0-82d3-52d40c5f6c77 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:26.085743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a400d6-930c-46d0-82d3-52d40c5f6c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:26.086018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a400d6-930c-46d0-82d3-52d40c5f6c77 None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:26.087251 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a400d6-930c-46d0-82d3-52d40c5f6c77 None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:26.088854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a400d6-930c-46d0-82d3-52d40c5f6c77 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:26.090948 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 192/366] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:25 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:26.091729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a400d6-930c-46d0-82d3-52d40c5f6c77 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:26.112650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03718fa3-8e29-49a6-ae80-5f656e4f1791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:26.113467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03718fa3-8e29-49a6-ae80-5f656e4f1791 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:26.113752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03718fa3-8e29-49a6-ae80-5f656e4f1791 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:26.114031 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03718fa3-8e29-49a6-ae80-5f656e4f1791 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:26.125854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03718fa3-8e29-49a6-ae80-5f656e4f1791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-03718fa3-8e29-49a6-ae80-5f656e4f1791 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:26.137169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:26.139287 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 175/367] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:59:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:26.141169 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03718fa3-8e29-49a6-ae80-5f656e4f1791 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:26.202439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a400d6-930c-46d0-82d3-52d40c5f6c77 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:26.240291 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d9828-6e14-4e43-8f55-49aac3a6c6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:26.240991 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d9828-6e14-4e43-8f55-49aac3a6c6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:26.241277 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d9828-6e14-4e43-8f55-49aac3a6c6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:26.241555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d9828-6e14-4e43-8f55-49aac3a6c6d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:26.242442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d9828-6e14-4e43-8f55-49aac3a6c6d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:26.242815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d9828-6e14-4e43-8f55-49aac3a6c6d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:26.244161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d9828-6e14-4e43-8f55-49aac3a6c6d4 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:26.254956 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 193/368] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:26.255726 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03718fa3-8e29-49a6-ae80-5f656e4f1791 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:26.255884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d9828-6e14-4e43-8f55-49aac3a6c6d4 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:26.293182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bdcd5-9b06-491e-b261-173f6ace5896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:26.293835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bdcd5-9b06-491e-b261-173f6ace5896 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:26.294123 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bdcd5-9b06-491e-b261-173f6ace5896 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:26.294447 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bdcd5-9b06-491e-b261-173f6ace5896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:26.294632 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:26.308671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:26.367413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d9828-6e14-4e43-8f55-49aac3a6c6d4 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:26.371925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bdcd5-9b06-491e-b261-173f6ace5896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:26.373704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bdcd5-9b06-491e-b261-173f6ace5896 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:26.403725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1b7bd-9044-4eca-8980-54ad4d0a9a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:26.404397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1b7bd-9044-4eca-8980-54ad4d0a9a1c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:26.404683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1b7bd-9044-4eca-8980-54ad4d0a9a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:26.404961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1b7bd-9044-4eca-8980-54ad4d0a9a1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:26.405787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1b7bd-9044-4eca-8980-54ad4d0a9a1c None admin] RBAC: Adding query filter params (group.id=d1cb6cdec2244e5bb303a3b659d83293, role.id=b10cd0627cc84a9c92dba333901e60e1, scope.project.id=36b45cb287dd49428bc291b9834d0b1e) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:26.406099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1b7bd-9044-4eca-8980-54ad4d0a9a1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:26.407602 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1b7bd-9044-4eca-8980-54ad4d0a9a1c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:26.413494 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 194/369] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jul 5 09:59:26 2022] GET /identity/v3/role_assignments?group.id=d1cb6cdec2244e5bb303a3b659d83293&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:26.414186 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1b7bd-9044-4eca-8980-54ad4d0a9a1c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:26.518185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:26.539395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1b7bd-9044-4eca-8980-54ad4d0a9a1c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:26.544878 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 176/370] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:26.545869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bdcd5-9b06-491e-b261-173f6ace5896 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:26.584489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3342c-fd83-45ff-bf25-80f87a3847fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:26.585197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3342c-fd83-45ff-bf25-80f87a3847fa None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:26.585480 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3342c-fd83-45ff-bf25-80f87a3847fa None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:26.585753 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3342c-fd83-45ff-bf25-80f87a3847fa None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:26.586544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3342c-fd83-45ff-bf25-80f87a3847fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:26.586852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3342c-fd83-45ff-bf25-80f87a3847fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:26.588291 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3342c-fd83-45ff-bf25-80f87a3847fa None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:26.607560 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 195/371] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:26.610604 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3342c-fd83-45ff-bf25-80f87a3847fa None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:26.628849 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:26.698423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bdcd5-9b06-491e-b261-173f6ace5896 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:26.741927 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0445e-cd6a-4891-9a1c-c27773eb9421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:26.742883 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0445e-cd6a-4891-9a1c-c27773eb9421 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:26.743616 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0445e-cd6a-4891-9a1c-c27773eb9421 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:26.743906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0445e-cd6a-4891-9a1c-c27773eb9421 None admin] PATH_INFO: `/v3/users/aecc43465e334fa49e2dca892a616405` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:26.745331 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0445e-cd6a-4891-9a1c-c27773eb9421 None admin] RBAC: Authorizing `identity:get_user(user_id=aecc43465e334fa49e2dca892a616405)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:26.749299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0445e-cd6a-4891-9a1c-c27773eb9421 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:26.751469 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 177/372] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 09:59:26 2022] GET /identity/v3/users/aecc43465e334fa49e2dca892a616405 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:26.752327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0445e-cd6a-4891-9a1c-c27773eb9421 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:26.875312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0445e-cd6a-4891-9a1c-c27773eb9421 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:26.883015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3342c-fd83-45ff-bf25-80f87a3847fa None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:26.901089 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:26.943957 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb05e0-1425-48e5-a270-83a60bb56e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:26.944994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb05e0-1425-48e5-a270-83a60bb56e77 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:26.945962 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb05e0-1425-48e5-a270-83a60bb56e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:26.946466 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb05e0-1425-48e5-a270-83a60bb56e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:26.952537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6488cb0-3e2f-4811-8d58-bd1ccd173f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:26.953716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6488cb0-3e2f-4811-8d58-bd1ccd173f6c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:26.959027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6488cb0-3e2f-4811-8d58-bd1ccd173f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:26.959540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6488cb0-3e2f-4811-8d58-bd1ccd173f6c None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:26.961034 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6488cb0-3e2f-4811-8d58-bd1ccd173f6c None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:26.962740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6488cb0-3e2f-4811-8d58-bd1ccd173f6c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:26.964869 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 196/373] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:26 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:26.968099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6488cb0-3e2f-4811-8d58-bd1ccd173f6c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:26.997212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:27.074188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb05e0-1425-48e5-a270-83a60bb56e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:27.076069 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb05e0-1425-48e5-a270-83a60bb56e77 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:27.149584 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6488cb0-3e2f-4811-8d58-bd1ccd173f6c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:27.227532 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c69f3-155d-4c21-a85e-ad0caf411c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:27.228213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c69f3-155d-4c21-a85e-ad0caf411c78 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:27.228500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c69f3-155d-4c21-a85e-ad0caf411c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:27.228775 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c69f3-155d-4c21-a85e-ad0caf411c78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:27.229636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c69f3-155d-4c21-a85e-ad0caf411c78 None admin] RBAC: Adding query filter params (role.id=8646f386c6704cb190f12a4b4e1a2245, user.id=aecc43465e334fa49e2dca892a616405, scope.system=all) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:27.229919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c69f3-155d-4c21-a85e-ad0caf411c78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:27.233611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c69f3-155d-4c21-a85e-ad0caf411c78 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:27.238712 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 178/374] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:27.240424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb05e0-1425-48e5-a270-83a60bb56e77 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:27.256596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:27.257742 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 197/375] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Jul 5 09:59:26 2022] GET /identity/v3/role_assignments?user.id=aecc43465e334fa49e2dca892a616405&scope.system=all&role.id=8646f386c6704cb190f12a4b4e1a2245 => generated 510 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:27.258486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c69f3-155d-4c21-a85e-ad0caf411c78 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:27.363693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb05e0-1425-48e5-a270-83a60bb56e77 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:27.406740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0817e5-be8c-4af1-9774-786fde59bb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:27.408013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0817e5-be8c-4af1-9774-786fde59bb1e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:27.408530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0817e5-be8c-4af1-9774-786fde59bb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:27.408982 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0817e5-be8c-4af1-9774-786fde59bb1e None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:27.410544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0817e5-be8c-4af1-9774-786fde59bb1e None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:27.412276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0817e5-be8c-4af1-9774-786fde59bb1e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:27.414481 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 179/376] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:27 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:27.420127 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0817e5-be8c-4af1-9774-786fde59bb1e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:27.450820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:27.511207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c69f3-155d-4c21-a85e-ad0caf411c78 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:27.578052 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64623e4d-63b0-4d09-a887-dd7e016df503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:27.578742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:27.581436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64623e4d-63b0-4d09-a887-dd7e016df503 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:27.582090 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64623e4d-63b0-4d09-a887-dd7e016df503 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:27.584021 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64623e4d-63b0-4d09-a887-dd7e016df503 None admin] PATH_INFO: `/v3/projects/95874cd0621e46bbbfb452bfbd474389` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:27.586363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64623e4d-63b0-4d09-a887-dd7e016df503 None admin] RBAC: Authorizing `identity:get_project(project_id=95874cd0621e46bbbfb452bfbd474389)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:27.588914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64623e4d-63b0-4d09-a887-dd7e016df503 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:27.591819 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 198/377] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:27 2022] GET /identity/v3/projects/95874cd0621e46bbbfb452bfbd474389 => generated 344 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:27.598755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64623e4d-63b0-4d09-a887-dd7e016df503 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:27.625673 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0817e5-be8c-4af1-9774-786fde59bb1e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:27.686464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddc0ae-3dff-4047-9d75-bdf218133fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:27.687758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddc0ae-3dff-4047-9d75-bdf218133fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:27.688991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddc0ae-3dff-4047-9d75-bdf218133fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:27.689443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddc0ae-3dff-4047-9d75-bdf218133fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:27.693308 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ddc0ae-3dff-4047-9d75-bdf218133fb1 None admin] RBAC: Adding query filter params (group.id=d1cb6cdec2244e5bb303a3b659d83293, role.id=70d38d0a968a4e9585c5873d24cfbbb1, scope.project.id=36b45cb287dd49428bc291b9834d0b1e) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:27.693815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ddc0ae-3dff-4047-9d75-bdf218133fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:27.695469 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ddc0ae-3dff-4047-9d75-bdf218133fb1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:27.704752 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 180/378] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jul 5 09:59:27 2022] GET /identity/v3/role_assignments?group.id=d1cb6cdec2244e5bb303a3b659d83293&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=70d38d0a968a4e9585c5873d24cfbbb1 => generated 260 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:27.770093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64623e4d-63b0-4d09-a887-dd7e016df503 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:27.814721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e8392-ccb6-4ee8-8ca0-57037b41deb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:27.815929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e8392-ccb6-4ee8-8ca0-57037b41deb6 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:27.816422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e8392-ccb6-4ee8-8ca0-57037b41deb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:27.816893 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e8392-ccb6-4ee8-8ca0-57037b41deb6 None admin] PATH_INFO: `/v3/groups/3780647aeafc4e17b1eb24faa4b8de66` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:27.823139 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e8392-ccb6-4ee8-8ca0-57037b41deb6 None admin] RBAC: Authorizing `identity:get_group(group_id=3780647aeafc4e17b1eb24faa4b8de66)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:27.824958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e8392-ccb6-4ee8-8ca0-57037b41deb6 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:27.827339 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 199/379] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:27 2022] GET /identity/v3/groups/3780647aeafc4e17b1eb24faa4b8de66 => generated 229 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:27.833223 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddc0ae-3dff-4047-9d75-bdf218133fb1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:27.950742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddc0ae-3dff-4047-9d75-bdf218133fb1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:27.991651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59875f89-b01d-447d-9de3-f1fb5a15650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:27.993608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b8202-3744-4998-ac46-e75dbc8167ad None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:27.994112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b8202-3744-4998-ac46-e75dbc8167ad None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:27.994594 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b8202-3744-4998-ac46-e75dbc8167ad None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:27.995710 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59875f89-b01d-447d-9de3-f1fb5a15650a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:27.995999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59875f89-b01d-447d-9de3-f1fb5a15650a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:27.996279 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59875f89-b01d-447d-9de3-f1fb5a15650a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:27.997205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59875f89-b01d-447d-9de3-f1fb5a15650a None admin] RBAC: Adding query filter params (group.id=3780647aeafc4e17b1eb24faa4b8de66, role.id=c184a177010144429f495f3290d28221, scope.project.id=95874cd0621e46bbbfb452bfbd474389) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:27.997427 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 200/380] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:27.997522 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59875f89-b01d-447d-9de3-f1fb5a15650a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:28.001906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59875f89-b01d-447d-9de3-f1fb5a15650a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:28.003595 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c8b01-858e-4919-a70c-4aee5843dc36 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:28.004102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c8b01-858e-4919-a70c-4aee5843dc36 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:28.004601 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c8b01-858e-4919-a70c-4aee5843dc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:28.006843 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 181/381] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jul 5 09:59:27 2022] GET /identity/v3/role_assignments?group.id=3780647aeafc4e17b1eb24faa4b8de66&scope.project.id=95874cd0621e46bbbfb452bfbd474389&role.id=c184a177010144429f495f3290d28221 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:28.011506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:28.025628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-759c8b01-858e-4919-a70c-4aee5843dc36 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:28.027427 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-759c8b01-858e-4919-a70c-4aee5843dc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:28.130828 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 201/382] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:28.156990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59875f89-b01d-447d-9de3-f1fb5a15650a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:28.159842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cc2ba-753d-40e8-b07f-9f8b2a4dcb79 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:28.160315 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cc2ba-753d-40e8-b07f-9f8b2a4dcb79 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:28.160766 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cc2ba-753d-40e8-b07f-9f8b2a4dcb79 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:28.162875 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 202/383] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:28.169022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252d0cd-876f-4cc8-a669-eaaebf3b52cb None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:28.169612 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252d0cd-876f-4cc8-a669-eaaebf3b52cb None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:28.170057 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252d0cd-876f-4cc8-a669-eaaebf3b52cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:28.175398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:28.204321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-0252d0cd-876f-4cc8-a669-eaaebf3b52cb None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:28.205612 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0252d0cd-876f-4cc8-a669-eaaebf3b52cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:28.312668 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 203/384] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:28.347877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59875f89-b01d-447d-9de3-f1fb5a15650a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:28.395506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252d0cd-876f-4cc8-a669-eaaebf3b52cb None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:28.409890 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8931ac-53a7-474b-97ff-094b2f1776f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:28.411006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8931ac-53a7-474b-97ff-094b2f1776f7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:28.411362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8931ac-53a7-474b-97ff-094b2f1776f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:28.413166 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8931ac-53a7-474b-97ff-094b2f1776f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:28.433399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8931ac-53a7-474b-97ff-094b2f1776f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-bf8931ac-53a7-474b-97ff-094b2f1776f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:28.441230 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:28.442165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:28.443348 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 182/385] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:28.451078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8931ac-53a7-474b-97ff-094b2f1776f7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:28.516058 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:28.533862 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:28.565216 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252d0cd-876f-4cc8-a669-eaaebf3b52cb None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:28.576155 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8931ac-53a7-474b-97ff-094b2f1776f7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:28.621755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a785a-5e7f-41e1-9e97-0480d3e193d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:28.622625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a785a-5e7f-41e1-9e97-0480d3e193d7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:28.622908 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a785a-5e7f-41e1-9e97-0480d3e193d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:28.623181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a785a-5e7f-41e1-9e97-0480d3e193d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:28.643286 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a785a-5e7f-41e1-9e97-0480d3e193d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:28.648647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d2f66-f2c8-4817-9308-d7a1d4e8c09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:28.649334 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d2f66-f2c8-4817-9308-d7a1d4e8c09c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:28.649668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d2f66-f2c8-4817-9308-d7a1d4e8c09c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:28.649956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d2f66-f2c8-4817-9308-d7a1d4e8c09c None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:28.650744 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d2f66-f2c8-4817-9308-d7a1d4e8c09c None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:28.651054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d2f66-f2c8-4817-9308-d7a1d4e8c09c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:28.652508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d2f66-f2c8-4817-9308-d7a1d4e8c09c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-e36a785a-5e7f-41e1-9e97-0480d3e193d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:28.662042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:28.664332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:28.666379 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 204/386] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:59:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:28.672919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a785a-5e7f-41e1-9e97-0480d3e193d7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:28.673578 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 183/387] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:28.678436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d2f66-f2c8-4817-9308-d7a1d4e8c09c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:28.780834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a785a-5e7f-41e1-9e97-0480d3e193d7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:28.805280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:28.828443 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefcc1b-0352-40db-b66d-b2efb0305dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:28.829552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefcc1b-0352-40db-b66d-b2efb0305dda None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:28.830031 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefcc1b-0352-40db-b66d-b2efb0305dda None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:28.830529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefcc1b-0352-40db-b66d-b2efb0305dda None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:28.832272 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefcc1b-0352-40db-b66d-b2efb0305dda None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:28.833870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefcc1b-0352-40db-b66d-b2efb0305dda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:28.835546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefcc1b-0352-40db-b66d-b2efb0305dda None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:28.846536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d2f66-f2c8-4817-9308-d7a1d4e8c09c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:28.856660 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 205/388] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:28.857866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefcc1b-0352-40db-b66d-b2efb0305dda None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:28.898898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:28.914612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbb817-9274-455d-ad1d-b3d1e09ec023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:28.915863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbb817-9274-455d-ad1d-b3d1e09ec023 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:28.916351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbb817-9274-455d-ad1d-b3d1e09ec023 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:28.916834 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbb817-9274-455d-ad1d-b3d1e09ec023 None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:28.920258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbb817-9274-455d-ad1d-b3d1e09ec023 None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:28.921917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbb817-9274-455d-ad1d-b3d1e09ec023 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:28.925290 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 184/389] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:28 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:28.931219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbb817-9274-455d-ad1d-b3d1e09ec023 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:29.023852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefcc1b-0352-40db-b66d-b2efb0305dda None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:29.060961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4bb37-bc09-4812-8b6e-b5a3e63cc1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:29.062193 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4bb37-bc09-4812-8b6e-b5a3e63cc1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:29.062739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4bb37-bc09-4812-8b6e-b5a3e63cc1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:29.063210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4bb37-bc09-4812-8b6e-b5a3e63cc1f5 None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:29.064626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4bb37-bc09-4812-8b6e-b5a3e63cc1f5 None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:29.066320 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4bb37-bc09-4812-8b6e-b5a3e63cc1f5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:29.068218 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 206/390] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:28 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:29.069730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1bafa-4e0b-4040-889c-15936ee080ef None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:29.070213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1bafa-4e0b-4040-889c-15936ee080ef None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:29.070712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1bafa-4e0b-4040-889c-15936ee080ef None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:29.072384 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 207/391] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:29.073655 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1bafa-4e0b-4040-889c-15936ee080ef None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:29.077143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbb817-9274-455d-ad1d-b3d1e09ec023 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:29.115616 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ced488-aec9-4888-b546-69662e26fc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:29.116859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ced488-aec9-4888-b546-69662e26fc26 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:29.117351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ced488-aec9-4888-b546-69662e26fc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:29.117837 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ced488-aec9-4888-b546-69662e26fc26 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:29.120983 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ced488-aec9-4888-b546-69662e26fc26 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:29.122704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ced488-aec9-4888-b546-69662e26fc26 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:29.124913 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 185/392] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:28 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:29.126403 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117d3ea-fa56-4439-926f-69c352109923 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:29.126895 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117d3ea-fa56-4439-926f-69c352109923 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:29.127435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117d3ea-fa56-4439-926f-69c352109923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:29.144947 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:29.148080 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2117d3ea-fa56-4439-926f-69c352109923 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:29.149450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2117d3ea-fa56-4439-926f-69c352109923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:29.211581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1bafa-4e0b-4040-889c-15936ee080ef None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:29.218778 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:29.228891 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 186/393] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:29.229920 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117d3ea-fa56-4439-926f-69c352109923 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:29.267612 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79582e3-12b1-473f-8e60-0a72428546c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:29.268903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79582e3-12b1-473f-8e60-0a72428546c2 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:29.269338 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79582e3-12b1-473f-8e60-0a72428546c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:29.269746 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79582e3-12b1-473f-8e60-0a72428546c2 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:29.272991 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79582e3-12b1-473f-8e60-0a72428546c2 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:29.274824 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79582e3-12b1-473f-8e60-0a72428546c2 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:29.277129 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 208/394] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:29 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:29.282814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79582e3-12b1-473f-8e60-0a72428546c2 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:29.339427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117d3ea-fa56-4439-926f-69c352109923 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:29.374975 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b0ce9-5129-4f06-a96f-0b34f50cd6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:29.375877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b0ce9-5129-4f06-a96f-0b34f50cd6d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:29.376162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b0ce9-5129-4f06-a96f-0b34f50cd6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:29.376470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b0ce9-5129-4f06-a96f-0b34f50cd6d0 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/groups/d1cb6cdec2244e5bb303a3b659d83293/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:29.379683 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b0ce9-5129-4f06-a96f-0b34f50cd6d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d72a2d59fc146d3afa2a17c4f4460f4, group_id=d1cb6cdec2244e5bb303a3b659d83293, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:29.381327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b0ce9-5129-4f06-a96f-0b34f50cd6d0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:29.389417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79582e3-12b1-473f-8e60-0a72428546c2 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:29.402529 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 187/395] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Jul 5 09:59:29 2022] PUT /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/groups/d1cb6cdec2244e5bb303a3b659d83293/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:29.404220 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b0ce9-5129-4f06-a96f-0b34f50cd6d0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:29.426199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:29.441578 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fde73-b279-4807-9362-fe513f42a25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:29.442984 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fde73-b279-4807-9362-fe513f42a25c None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:29.443271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fde73-b279-4807-9362-fe513f42a25c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:29.443544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fde73-b279-4807-9362-fe513f42a25c None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/groups/d1cb6cdec2244e5bb303a3b659d83293/roles/70d38d0a968a4e9585c5873d24cfbbb1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:29.446691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fde73-b279-4807-9362-fe513f42a25c None admin] RBAC: Authorizing `identity:create_grant(project_id=1d72a2d59fc146d3afa2a17c4f4460f4, group_id=d1cb6cdec2244e5bb303a3b659d83293, role_id=70d38d0a968a4e9585c5873d24cfbbb1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:29.448460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fde73-b279-4807-9362-fe513f42a25c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:29.469599 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 209/396] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Jul 5 09:59:29 2022] PUT /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4/groups/d1cb6cdec2244e5bb303a3b659d83293/roles/70d38d0a968a4e9585c5873d24cfbbb1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 09:59:29.482961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:29.542265 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b0ce9-5129-4f06-a96f-0b34f50cd6d0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:29.589020 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d3fdb-2e9c-4254-bbcb-e360f43add21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:29.589984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d3fdb-2e9c-4254-bbcb-e360f43add21 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:29.590215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d3fdb-2e9c-4254-bbcb-e360f43add21 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:29.590518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d3fdb-2e9c-4254-bbcb-e360f43add21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:29.607263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d3fdb-2e9c-4254-bbcb-e360f43add21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-2e6d3fdb-2e9c-4254-bbcb-e360f43add21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:29.615756 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:29.616722 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:29.617502 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 188/397] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:29.627505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fde73-b279-4807-9362-fe513f42a25c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:29.761907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fde73-b279-4807-9362-fe513f42a25c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:29.802613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a38e9-8770-43df-b494-7fc6baf10e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:29.803307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a38e9-8770-43df-b494-7fc6baf10e18 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:29.803597 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a38e9-8770-43df-b494-7fc6baf10e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:29.803937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a38e9-8770-43df-b494-7fc6baf10e18 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:29.804637 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a38e9-8770-43df-b494-7fc6baf10e18 None admin] RBAC: Adding query filter params (name=member) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:29.804943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a38e9-8770-43df-b494-7fc6baf10e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:29.806389 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a38e9-8770-43df-b494-7fc6baf10e18 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:29.817862 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 210/398] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:29.823312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d3fdb-2e9c-4254-bbcb-e360f43add21 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:29.961789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d3fdb-2e9c-4254-bbcb-e360f43add21 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:29.978991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:30.009790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abba39b-76e7-45e5-9e1f-28c1e724659b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:30.010744 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba39b-76e7-45e5-9e1f-28c1e724659b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:30.011029 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba39b-76e7-45e5-9e1f-28c1e724659b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:30.011311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba39b-76e7-45e5-9e1f-28c1e724659b None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:30.012523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abba39b-76e7-45e5-9e1f-28c1e724659b None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:30.013970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abba39b-76e7-45e5-9e1f-28c1e724659b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:30.016947 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 189/399] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:29 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:30.021816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a38e9-8770-43df-b494-7fc6baf10e18 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:30.093900 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:30.163112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a38e9-8770-43df-b494-7fc6baf10e18 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:30.205433 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df19483-a534-49d5-bb76-6d322907a841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:30.206752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df19483-a534-49d5-bb76-6d322907a841 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:30.207301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df19483-a534-49d5-bb76-6d322907a841 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:30.208199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df19483-a534-49d5-bb76-6d322907a841 None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:30.209668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df19483-a534-49d5-bb76-6d322907a841 None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:30.211385 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df19483-a534-49d5-bb76-6d322907a841 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:30.213758 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 211/400] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:30 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:30.223443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abba39b-76e7-45e5-9e1f-28c1e724659b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:30.272226 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a31f5b-c14b-4dab-b1a1-fac4b49151cc None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:30.272577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a31f5b-c14b-4dab-b1a1-fac4b49151cc None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:30.272786 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a31f5b-c14b-4dab-b1a1-fac4b49151cc None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:30.274400 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 212/401] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:30.280833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57337ff-5d58-4150-bdf2-e1c9f7ba00c8 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:30.281120 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57337ff-5d58-4150-bdf2-e1c9f7ba00c8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:30.281395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57337ff-5d58-4150-bdf2-e1c9f7ba00c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:30.303583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-d57337ff-5d58-4150-bdf2-e1c9f7ba00c8 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:30.304483 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d57337ff-5d58-4150-bdf2-e1c9f7ba00c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:30.400323 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abba39b-76e7-45e5-9e1f-28c1e724659b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:30.411936 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 213/402] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:30.438163 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57337ff-5d58-4150-bdf2-e1c9f7ba00c8 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:30.445262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9647a-f5b6-4c2d-890d-f4f2835a0307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:30.446367 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9647a-f5b6-4c2d-890d-f4f2835a0307 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:30.446810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9647a-f5b6-4c2d-890d-f4f2835a0307 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:30.447234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9647a-f5b6-4c2d-890d-f4f2835a0307 None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e/groups/d1cb6cdec2244e5bb303a3b659d83293/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:30.453261 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9647a-f5b6-4c2d-890d-f4f2835a0307 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b45cb287dd49428bc291b9834d0b1e, group_id=d1cb6cdec2244e5bb303a3b659d83293, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:30.455132 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9647a-f5b6-4c2d-890d-f4f2835a0307 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:30.480164 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 190/403] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Jul 5 09:59:30 2022] PUT /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e/groups/d1cb6cdec2244e5bb303a3b659d83293/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:30.505313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:30.582694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57337ff-5d58-4150-bdf2-e1c9f7ba00c8 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:30.634584 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d49668-ffd7-4af9-ac9e-788023eba6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:30.635434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d49668-ffd7-4af9-ac9e-788023eba6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:30.635718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d49668-ffd7-4af9-ac9e-788023eba6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:30.636001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d49668-ffd7-4af9-ac9e-788023eba6f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:30.657844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d49668-ffd7-4af9-ac9e-788023eba6f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-42d49668-ffd7-4af9-ac9e-788023eba6f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:30.666374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:30.667319 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:30.668600 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 214/404] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:59:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:30.674052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9647a-f5b6-4c2d-890d-f4f2835a0307 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:30.749900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:30.828680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9647a-f5b6-4c2d-890d-f4f2835a0307 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:30.863058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f2233-13dc-4bb4-bdb0-acb689def126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:30.863740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f2233-13dc-4bb4-bdb0-acb689def126 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:30.864059 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f2233-13dc-4bb4-bdb0-acb689def126 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:30.864303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f2233-13dc-4bb4-bdb0-acb689def126 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:30.865122 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f2233-13dc-4bb4-bdb0-acb689def126 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:30.865431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f2233-13dc-4bb4-bdb0-acb689def126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:30.866849 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f2233-13dc-4bb4-bdb0-acb689def126 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:30.875297 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 191/405] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:30.880937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d49668-ffd7-4af9-ac9e-788023eba6f9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:30.892138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6158bf-7878-4c0a-9b2f-5d34800994da None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:30.892438 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6158bf-7878-4c0a-9b2f-5d34800994da None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:30.892696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6158bf-7878-4c0a-9b2f-5d34800994da None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:30.894424 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 192/406] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 09:59:30.899263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a4e06-1b70-4937-8dbf-0f8a2e642c97 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:30.899551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a4e06-1b70-4937-8dbf-0f8a2e642c97 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:30.899827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a4e06-1b70-4937-8dbf-0f8a2e642c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:30.922190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cd6a4e06-1b70-4937-8dbf-0f8a2e642c97 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:30.923445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cd6a4e06-1b70-4937-8dbf-0f8a2e642c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:30.997091 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 193/407] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:31.021374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a4e06-1b70-4937-8dbf-0f8a2e642c97 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:31.058313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d49668-ffd7-4af9-ac9e-788023eba6f9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:31.059011 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:31.062191 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:31.127116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc3f0f-12fa-4674-8e7f-99f9d08f8e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:31.128016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc3f0f-12fa-4674-8e7f-99f9d08f8e92 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:31.128282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc3f0f-12fa-4674-8e7f-99f9d08f8e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:31.128646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc3f0f-12fa-4674-8e7f-99f9d08f8e92 None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:31.129948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc3f0f-12fa-4674-8e7f-99f9d08f8e92 None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:31.131517 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc3f0f-12fa-4674-8e7f-99f9d08f8e92 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:31.138388 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a4e06-1b70-4937-8dbf-0f8a2e642c97 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:31.142418 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 215/408] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:30 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:31.143357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc3f0f-12fa-4674-8e7f-99f9d08f8e92 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:31.175177 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b562414-baea-489a-8cd7-0aefdde2d38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:31.176364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b562414-baea-489a-8cd7-0aefdde2d38c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:31.176650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b562414-baea-489a-8cd7-0aefdde2d38c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:31.176979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b562414-baea-489a-8cd7-0aefdde2d38c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:31.188543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b562414-baea-489a-8cd7-0aefdde2d38c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-7b562414-baea-489a-8cd7-0aefdde2d38c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:31.197165 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:31.198079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:31.198890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:31.203840 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 194/409] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:59:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:31.205952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b562414-baea-489a-8cd7-0aefdde2d38c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:31.310722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:31.313714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b562414-baea-489a-8cd7-0aefdde2d38c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:31.352408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41fc88-8568-4d93-9733-3dc120bb428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:31.353091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41fc88-8568-4d93-9733-3dc120bb428c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:31.353378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41fc88-8568-4d93-9733-3dc120bb428c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:31.353656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41fc88-8568-4d93-9733-3dc120bb428c None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:31.354530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41fc88-8568-4d93-9733-3dc120bb428c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:31.354837 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41fc88-8568-4d93-9733-3dc120bb428c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:31.356524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41fc88-8568-4d93-9733-3dc120bb428c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:31.360262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc3f0f-12fa-4674-8e7f-99f9d08f8e92 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:31.367334 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 195/410] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:31.372809 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41fc88-8568-4d93-9733-3dc120bb428c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:31.424238 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94069523-f684-4085-ae24-b73d8671d05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:31.425121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94069523-f684-4085-ae24-b73d8671d05c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:31.425405 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94069523-f684-4085-ae24-b73d8671d05c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:31.425682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94069523-f684-4085-ae24-b73d8671d05c None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:31.429863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94069523-f684-4085-ae24-b73d8671d05c None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:31.431222 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94069523-f684-4085-ae24-b73d8671d05c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:31.432963 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 216/411] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:31 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:31.439269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94069523-f684-4085-ae24-b73d8671d05c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:31.518624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:31.537462 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41fc88-8568-4d93-9733-3dc120bb428c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:31.580804 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e51f4-6bf1-4b3a-a354-03e3a4c43130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:31.581594 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e51f4-6bf1-4b3a-a354-03e3a4c43130 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:31.584496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e51f4-6bf1-4b3a-a354-03e3a4c43130 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:31.584791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e51f4-6bf1-4b3a-a354-03e3a4c43130 None admin] PATH_INFO: `/v3/groups/3780647aeafc4e17b1eb24faa4b8de66` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:31.589416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e51f4-6bf1-4b3a-a354-03e3a4c43130 None admin] RBAC: Authorizing `identity:get_group(group_id=3780647aeafc4e17b1eb24faa4b8de66)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:31.590903 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e51f4-6bf1-4b3a-a354-03e3a4c43130 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:31.592567 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 196/412] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:31 2022] GET /identity/v3/groups/3780647aeafc4e17b1eb24faa4b8de66 => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:31.598656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e51f4-6bf1-4b3a-a354-03e3a4c43130 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:31.643714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94069523-f684-4085-ae24-b73d8671d05c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:31.650584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:31.683874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23916193-213e-4b2f-ab30-517af3510888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:31.684948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23916193-213e-4b2f-ab30-517af3510888 None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:31.685235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23916193-213e-4b2f-ab30-517af3510888 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:31.685512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23916193-213e-4b2f-ab30-517af3510888 None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e/groups/d1cb6cdec2244e5bb303a3b659d83293/roles/70d38d0a968a4e9585c5873d24cfbbb1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:31.687878 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23916193-213e-4b2f-ab30-517af3510888 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b45cb287dd49428bc291b9834d0b1e, group_id=d1cb6cdec2244e5bb303a3b659d83293, role_id=70d38d0a968a4e9585c5873d24cfbbb1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:31.689334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23916193-213e-4b2f-ab30-517af3510888 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:31.708121 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 217/413] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Jul 5 09:59:31 2022] PUT /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e/groups/d1cb6cdec2244e5bb303a3b659d83293/roles/70d38d0a968a4e9585c5873d24cfbbb1 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:31.836403 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e51f4-6bf1-4b3a-a354-03e3a4c43130 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:31.886327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e3378-a542-41a0-9749-2688e6bbcf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:31.887205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3378-a542-41a0-9749-2688e6bbcf0c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:31.887485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3378-a542-41a0-9749-2688e6bbcf0c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:31.887758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3378-a542-41a0-9749-2688e6bbcf0c None admin] PATH_INFO: `/v3/projects/95874cd0621e46bbbfb452bfbd474389` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:31.888985 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e3378-a542-41a0-9749-2688e6bbcf0c None admin] RBAC: Authorizing `identity:get_project(project_id=95874cd0621e46bbbfb452bfbd474389)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:31.890443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e3378-a542-41a0-9749-2688e6bbcf0c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:31.893965 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 197/414] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:31 2022] GET /identity/v3/projects/95874cd0621e46bbbfb452bfbd474389 => generated 344 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:31.899937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23916193-213e-4b2f-ab30-517af3510888 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:32.020045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23916193-213e-4b2f-ab30-517af3510888 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:32.022513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:32.066427 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750c9a2-6d7a-4522-a062-800123727bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:32.067339 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750c9a2-6d7a-4522-a062-800123727bce None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:32.067622 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750c9a2-6d7a-4522-a062-800123727bce None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:32.067939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750c9a2-6d7a-4522-a062-800123727bce None admin] PATH_INFO: `/v3/projects/95874cd0621e46bbbfb452bfbd474389/groups/3780647aeafc4e17b1eb24faa4b8de66/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:32.069960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750c9a2-6d7a-4522-a062-800123727bce None admin] RBAC: Authorizing `identity:create_grant(project_id=95874cd0621e46bbbfb452bfbd474389, group_id=3780647aeafc4e17b1eb24faa4b8de66, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:32.071451 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750c9a2-6d7a-4522-a062-800123727bce None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:32.090573 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 218/415] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Jul 5 09:59:31 2022] PUT /identity/v3/projects/95874cd0621e46bbbfb452bfbd474389/groups/3780647aeafc4e17b1eb24faa4b8de66/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:32.176789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bb579-8e0e-41cb-96c4-5334637b25cb None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:32.177273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bb579-8e0e-41cb-96c4-5334637b25cb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:32.177441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bb579-8e0e-41cb-96c4-5334637b25cb None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:32.179275 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 198/416] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:32.184913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340e1fe-d67d-4c36-b964-ad6560ce5294 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:32.185200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340e1fe-d67d-4c36-b964-ad6560ce5294 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:32.185478 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340e1fe-d67d-4c36-b964-ad6560ce5294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:32.203568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a340e1fe-d67d-4c36-b964-ad6560ce5294 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:32.204856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a340e1fe-d67d-4c36-b964-ad6560ce5294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:32.251888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831d865-81fd-4c68-89ba-59ed130f7125 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:32.252181 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831d865-81fd-4c68-89ba-59ed130f7125 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:32.252518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831d865-81fd-4c68-89ba-59ed130f7125 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:32.253946 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 199/417] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:32.266164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd53581-5004-4a85-b837-1b732869f0e5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:32.266543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd53581-5004-4a85-b837-1b732869f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:32.266767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd53581-5004-4a85-b837-1b732869f0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:32.291469 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 219/418] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:32.295398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0bd53581-5004-4a85-b837-1b732869f0e5 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:32.296629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0bd53581-5004-4a85-b837-1b732869f0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:32.320646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340e1fe-d67d-4c36-b964-ad6560ce5294 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:32.385337 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 200/419] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:32.408863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd53581-5004-4a85-b837-1b732869f0e5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:32.419305 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:32.432434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340e1fe-d67d-4c36-b964-ad6560ce5294 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:32.437921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:32.470108 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47024077-3fed-457b-8941-f27e3ef06299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:32.470982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47024077-3fed-457b-8941-f27e3ef06299 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:32.471268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47024077-3fed-457b-8941-f27e3ef06299 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:32.471547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47024077-3fed-457b-8941-f27e3ef06299 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:32.501195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47024077-3fed-457b-8941-f27e3ef06299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-47024077-3fed-457b-8941-f27e3ef06299 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:32.509599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:32.510787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:32.511966 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 220/420] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:59:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:32.518601 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47024077-3fed-457b-8941-f27e3ef06299 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:32.548037 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd53581-5004-4a85-b837-1b732869f0e5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:32.623635 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac4be1-a055-4c08-a3a6-1b995b506fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:32.624713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac4be1-a055-4c08-a3a6-1b995b506fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:32.625091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac4be1-a055-4c08-a3a6-1b995b506fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:32.625465 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac4be1-a055-4c08-a3a6-1b995b506fc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:32.654986 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47024077-3fed-457b-8941-f27e3ef06299 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:32.657281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac4be1-a055-4c08-a3a6-1b995b506fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-aeac4be1-a055-4c08-a3a6-1b995b506fc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:32.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:32.666345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:32.667305 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:32.674885 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 201/421] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:32.687258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac4be1-a055-4c08-a3a6-1b995b506fc3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:32.699404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db62ae-164d-4f25-be96-6f5cf070deb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:32.700280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db62ae-164d-4f25-be96-6f5cf070deb6 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:32.700734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db62ae-164d-4f25-be96-6f5cf070deb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:32.701151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db62ae-164d-4f25-be96-6f5cf070deb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:32.702033 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db62ae-164d-4f25-be96-6f5cf070deb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:32.702499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db62ae-164d-4f25-be96-6f5cf070deb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:32.704146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db62ae-164d-4f25-be96-6f5cf070deb6 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:32.713821 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 221/422] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:32.719862 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db62ae-164d-4f25-be96-6f5cf070deb6 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:32.733093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:32.826627 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db62ae-164d-4f25-be96-6f5cf070deb6 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:32.862159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c7e1d-8356-49ab-91fa-1a3a9c83fa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:32.862889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c7e1d-8356-49ab-91fa-1a3a9c83fa2d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:32.863173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c7e1d-8356-49ab-91fa-1a3a9c83fa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:32.863441 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c7e1d-8356-49ab-91fa-1a3a9c83fa2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:32.911510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:32.936570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c7e1d-8356-49ab-91fa-1a3a9c83fa2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:32.938041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c7e1d-8356-49ab-91fa-1a3a9c83fa2d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:32.949882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:32.965081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac4be1-a055-4c08-a3a6-1b995b506fc3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:33.027924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dc454-d7a1-4643-8ec6-c00127ff07e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:33.028643 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dc454-d7a1-4643-8ec6-c00127ff07e3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:33.028933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dc454-d7a1-4643-8ec6-c00127ff07e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:33.029214 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dc454-d7a1-4643-8ec6-c00127ff07e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:33.029997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dc454-d7a1-4643-8ec6-c00127ff07e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:33.030338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dc454-d7a1-4643-8ec6-c00127ff07e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:33.031748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dc454-d7a1-4643-8ec6-c00127ff07e3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:33.042484 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 202/423] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:33.048168 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dc454-d7a1-4643-8ec6-c00127ff07e3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:33.092855 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 222/424] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:33.098444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c7e1d-8356-49ab-91fa-1a3a9c83fa2d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:33.228491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dc454-d7a1-4643-8ec6-c00127ff07e3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:33.234282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c7e1d-8356-49ab-91fa-1a3a9c83fa2d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:33.246700 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:33.287245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b4f9c-aab1-45ac-b1a8-dea525c31760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:33.287915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b4f9c-aab1-45ac-b1a8-dea525c31760 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:33.288204 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b4f9c-aab1-45ac-b1a8-dea525c31760 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:33.288485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b4f9c-aab1-45ac-b1a8-dea525c31760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:33.289649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b73ae-fc32-4ee2-adb1-ba1a254fab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:33.290552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b73ae-fc32-4ee2-adb1-ba1a254fab70 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:33.290833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b73ae-fc32-4ee2-adb1-ba1a254fab70 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:33.291108 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b73ae-fc32-4ee2-adb1-ba1a254fab70 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:33.294179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b73ae-fc32-4ee2-adb1-ba1a254fab70 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:33.295661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b73ae-fc32-4ee2-adb1-ba1a254fab70 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:33.299078 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 223/425] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:33 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:33.302708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b73ae-fc32-4ee2-adb1-ba1a254fab70 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:33.320601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:33.409049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b4f9c-aab1-45ac-b1a8-dea525c31760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:33.411753 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b4f9c-aab1-45ac-b1a8-dea525c31760 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:33.447657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b73ae-fc32-4ee2-adb1-ba1a254fab70 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:33.491969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232cb18-8264-44b3-ab68-2c38a336108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:33.493119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232cb18-8264-44b3-ab68-2c38a336108c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:33.493453 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232cb18-8264-44b3-ab68-2c38a336108c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:33.494203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232cb18-8264-44b3-ab68-2c38a336108c None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:33.496115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232cb18-8264-44b3-ab68-2c38a336108c None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:33.498198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232cb18-8264-44b3-ab68-2c38a336108c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:33.502306 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 224/426] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:33 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:33.503720 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780d5ff-84dc-4a21-888b-91c32e88e3c9 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:33.504013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780d5ff-84dc-4a21-888b-91c32e88e3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:33.504290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780d5ff-84dc-4a21-888b-91c32e88e3c9 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:33.506044 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 225/427] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:33.508292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780d5ff-84dc-4a21-888b-91c32e88e3c9 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:33.557172 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 203/428] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:33.559462 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df237fce-586a-40bb-a081-e8c29605bc7c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:33.559761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df237fce-586a-40bb-a081-e8c29605bc7c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:33.560042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df237fce-586a-40bb-a081-e8c29605bc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:33.572128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:33.579916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-df237fce-586a-40bb-a081-e8c29605bc7c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:33.581139 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-df237fce-586a-40bb-a081-e8c29605bc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:33.632482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:33.644007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780d5ff-84dc-4a21-888b-91c32e88e3c9 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:33.664627 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 204/429] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:33.665747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df237fce-586a-40bb-a081-e8c29605bc7c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:33.686179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa43cc-e1a7-4b89-be19-82c01d432cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:33.687734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa43cc-e1a7-4b89-be19-82c01d432cfc None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:33.688255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa43cc-e1a7-4b89-be19-82c01d432cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:33.688718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa43cc-e1a7-4b89-be19-82c01d432cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:33.689733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa43cc-e1a7-4b89-be19-82c01d432cfc None admin] RBAC: Adding query filter params (group.id=d1cb6cdec2244e5bb303a3b659d83293, role.id=70d38d0a968a4e9585c5873d24cfbbb1, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:33.690274 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa43cc-e1a7-4b89-be19-82c01d432cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:33.692804 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa43cc-e1a7-4b89-be19-82c01d432cfc None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:33.709505 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 226/430] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jul 5 09:59:33 2022] GET /identity/v3/role_assignments?group.id=d1cb6cdec2244e5bb303a3b659d83293&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=70d38d0a968a4e9585c5873d24cfbbb1 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:33.710547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa43cc-e1a7-4b89-be19-82c01d432cfc None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:33.857988 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df237fce-586a-40bb-a081-e8c29605bc7c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:33.909704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db517401-741b-4bbc-b5b1-9c5bc0dae216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:33.910619 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db517401-741b-4bbc-b5b1-9c5bc0dae216 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:33.910907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db517401-741b-4bbc-b5b1-9c5bc0dae216 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:33.911185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db517401-741b-4bbc-b5b1-9c5bc0dae216 None admin] PATH_INFO: `/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:33.911672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa43cc-e1a7-4b89-be19-82c01d432cfc None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:33.912566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db517401-741b-4bbc-b5b1-9c5bc0dae216 None admin] RBAC: Authorizing `identity:get_project(project_id=1d72a2d59fc146d3afa2a17c4f4460f4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:33.914083 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db517401-741b-4bbc-b5b1-9c5bc0dae216 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:33.916199 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 205/431] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:33 2022] GET /identity/v3/projects/1d72a2d59fc146d3afa2a17c4f4460f4 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:33.922523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db517401-741b-4bbc-b5b1-9c5bc0dae216 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:33.925310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:33.956654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145d7f0-0db1-41dc-87d2-f911e0991a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:33.958804 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145d7f0-0db1-41dc-87d2-f911e0991a54 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:33.959279 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145d7f0-0db1-41dc-87d2-f911e0991a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:33.959954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145d7f0-0db1-41dc-87d2-f911e0991a54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:33.972736 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:33.981576 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145d7f0-0db1-41dc-87d2-f911e0991a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-c145d7f0-0db1-41dc-87d2-f911e0991a54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:33.988952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:33.989859 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:33.995019 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 227/432] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:34.000200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145d7f0-0db1-41dc-87d2-f911e0991a54 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:34.050605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db517401-741b-4bbc-b5b1-9c5bc0dae216 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:34.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e22012-be34-48ae-8b8c-70bf652607bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:34.091116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22012-be34-48ae-8b8c-70bf652607bf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.091517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22012-be34-48ae-8b8c-70bf652607bf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.091798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22012-be34-48ae-8b8c-70bf652607bf None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.093012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e22012-be34-48ae-8b8c-70bf652607bf None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:34.094719 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e22012-be34-48ae-8b8c-70bf652607bf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:34.096672 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 206/433] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:33 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:34.097981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d33b4c-0776-4898-a6c2-2df761e44335 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.098293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d33b4c-0776-4898-a6c2-2df761e44335 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.098576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d33b4c-0776-4898-a6c2-2df761e44335 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.100037 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 207/434] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:34.103312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d33b4c-0776-4898-a6c2-2df761e44335 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:34.113356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145d7f0-0db1-41dc-87d2-f911e0991a54 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:34.148720 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0660f-5f80-43fd-b137-73eb58137c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:34.149495 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0660f-5f80-43fd-b137-73eb58137c56 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.149833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0660f-5f80-43fd-b137-73eb58137c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.150084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0660f-5f80-43fd-b137-73eb58137c56 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.150825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0660f-5f80-43fd-b137-73eb58137c56 None admin] RBAC: Adding query filter params (name=member) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:34.151134 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0660f-5f80-43fd-b137-73eb58137c56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:34.152527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0660f-5f80-43fd-b137-73eb58137c56 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:34.161672 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 228/435] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 09:59:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:34.163069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0c94-537b-460a-b5a2-eaeb1d0fd658 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.163359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0c94-537b-460a-b5a2-eaeb1d0fd658 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.163647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0c94-537b-460a-b5a2-eaeb1d0fd658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.180894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-1b8e0c94-537b-460a-b5a2-eaeb1d0fd658 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:34.182101 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1b8e0c94-537b-460a-b5a2-eaeb1d0fd658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:34.209391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:34.214421 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d33b4c-0776-4898-a6c2-2df761e44335 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:34.226148 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:34.251318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87f8fa-70f9-4dff-81c5-c52d7199104f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:34.251980 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87f8fa-70f9-4dff-81c5-c52d7199104f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.252263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87f8fa-70f9-4dff-81c5-c52d7199104f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.252544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87f8fa-70f9-4dff-81c5-c52d7199104f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.253377 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87f8fa-70f9-4dff-81c5-c52d7199104f None admin] RBAC: Adding query filter params (group.id=d1cb6cdec2244e5bb303a3b659d83293, role.id=b10cd0627cc84a9c92dba333901e60e1, scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:34.253681 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87f8fa-70f9-4dff-81c5-c52d7199104f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:34.255144 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87f8fa-70f9-4dff-81c5-c52d7199104f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:34.258167 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 229/436] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:34.259251 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e0c94-537b-460a-b5a2-eaeb1d0fd658 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:34.272389 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 208/437] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jul 5 09:59:34 2022] GET /identity/v3/role_assignments?group.id=d1cb6cdec2244e5bb303a3b659d83293&scope.project.id=1d72a2d59fc146d3afa2a17c4f4460f4&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:34.307205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87f8fa-70f9-4dff-81c5-c52d7199104f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:34.395697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e0c94-537b-460a-b5a2-eaeb1d0fd658 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:34.428116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87f8fa-70f9-4dff-81c5-c52d7199104f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:34.444318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6bddc-4e69-40e0-ab1c-0fa43e4f1d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:34.445373 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6bddc-4e69-40e0-ab1c-0fa43e4f1d96 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.445866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6bddc-4e69-40e0-ab1c-0fa43e4f1d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.446212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6bddc-4e69-40e0-ab1c-0fa43e4f1d96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.467728 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bc8c8-fb97-4b10-b3c5-e02b34af676d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:34.468726 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bc8c8-fb97-4b10-b3c5-e02b34af676d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.469102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bc8c8-fb97-4b10-b3c5-e02b34af676d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.469417 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bc8c8-fb97-4b10-b3c5-e02b34af676d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.486665 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bc8c8-fb97-4b10-b3c5-e02b34af676d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:34.488024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9a8bc8c8-fb97-4b10-b3c5-e02b34af676d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:34.495051 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:34.496124 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:34.497156 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 209/438] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:59:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:34.498024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1728668-71ce-4fe1-86d9-b336e9ac05c5 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.498573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1728668-71ce-4fe1-86d9-b336e9ac05c5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.498682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1728668-71ce-4fe1-86d9-b336e9ac05c5 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.500496 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 210/439] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:34.502479 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1728668-71ce-4fe1-86d9-b336e9ac05c5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:34.541215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6bddc-4e69-40e0-ab1c-0fa43e4f1d96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:34.543068 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6bddc-4e69-40e0-ab1c-0fa43e4f1d96 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:34.550934 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:34.625263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1728668-71ce-4fe1-86d9-b336e9ac05c5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:34.660687 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 230/440] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:34.662714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e121598-3633-4e4a-8b12-6f2ebeb79d37 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.662929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e121598-3633-4e4a-8b12-6f2ebeb79d37 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.663449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e121598-3633-4e4a-8b12-6f2ebeb79d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.675208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b17bf-461d-42e8-8561-a8c20ae95041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:34.675916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b17bf-461d-42e8-8561-a8c20ae95041 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.676241 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b17bf-461d-42e8-8561-a8c20ae95041 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.676490 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b17bf-461d-42e8-8561-a8c20ae95041 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.677298 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b17bf-461d-42e8-8561-a8c20ae95041 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:34.677608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b17bf-461d-42e8-8561-a8c20ae95041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:34.679070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b17bf-461d-42e8-8561-a8c20ae95041 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:34.683306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-0e121598-3633-4e4a-8b12-6f2ebeb79d37 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:34.685025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0e121598-3633-4e4a-8b12-6f2ebeb79d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:34.688649 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 211/441] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:34.688874 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b17bf-461d-42e8-8561-a8c20ae95041 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:34.756806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:34.771044 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 231/442] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:34.772125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e121598-3633-4e4a-8b12-6f2ebeb79d37 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:34.812558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:34.820546 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b17bf-461d-42e8-8561-a8c20ae95041 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:34.855734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b3d52-437d-4e49-95ee-8d8f02cf0f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:34.856611 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b3d52-437d-4e49-95ee-8d8f02cf0f2f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.856889 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b3d52-437d-4e49-95ee-8d8f02cf0f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.857162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b3d52-437d-4e49-95ee-8d8f02cf0f2f None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.858363 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b3d52-437d-4e49-95ee-8d8f02cf0f2f None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:34.859800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b3d52-437d-4e49-95ee-8d8f02cf0f2f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:34.861475 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 212/443] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:34 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:34.862258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b3d52-437d-4e49-95ee-8d8f02cf0f2f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:34.876912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e121598-3633-4e4a-8b12-6f2ebeb79d37 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:34.909851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644af1d-e51d-4c2d-8b04-da7283efdd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:34.910570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644af1d-e51d-4c2d-8b04-da7283efdd3e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.910850 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644af1d-e51d-4c2d-8b04-da7283efdd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.911127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644af1d-e51d-4c2d-8b04-da7283efdd3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:34.958995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b3d52-437d-4e49-95ee-8d8f02cf0f2f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:34.967531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:34.976842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644af1d-e51d-4c2d-8b04-da7283efdd3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:34.978353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644af1d-e51d-4c2d-8b04-da7283efdd3e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:34.991649 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fca17-88af-423d-98ef-782239b7d7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:34.992430 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fca17-88af-423d-98ef-782239b7d7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:34.992722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fca17-88af-423d-98ef-782239b7d7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:34.992988 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fca17-88af-423d-98ef-782239b7d7b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:35.003110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fca17-88af-423d-98ef-782239b7d7b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-2b2fca17-88af-423d-98ef-782239b7d7b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:35.011223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:35.012112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 213/444] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:59:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:35.013542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fca17-88af-423d-98ef-782239b7d7b0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:35.044691 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:35.073409 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 232/445] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:35.074374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644af1d-e51d-4c2d-8b04-da7283efdd3e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:35.121140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fca17-88af-423d-98ef-782239b7d7b0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:35.153960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130cc30-ed6e-4648-a0e1-96f65207a440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:35.154833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130cc30-ed6e-4648-a0e1-96f65207a440 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:35.155114 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130cc30-ed6e-4648-a0e1-96f65207a440 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:35.155433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130cc30-ed6e-4648-a0e1-96f65207a440 None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:35.156648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130cc30-ed6e-4648-a0e1-96f65207a440 None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:35.158100 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130cc30-ed6e-4648-a0e1-96f65207a440 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:35.159744 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 214/446] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:35 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:35.160506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130cc30-ed6e-4648-a0e1-96f65207a440 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:35.163403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:35.173843 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644af1d-e51d-4c2d-8b04-da7283efdd3e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:35.206644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0ca6d-69a9-4588-99b7-d3f6177af908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:35.207337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0ca6d-69a9-4588-99b7-d3f6177af908 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:35.207616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0ca6d-69a9-4588-99b7-d3f6177af908 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:35.208249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0ca6d-69a9-4588-99b7-d3f6177af908 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:35.209081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0ca6d-69a9-4588-99b7-d3f6177af908 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:35.209357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0ca6d-69a9-4588-99b7-d3f6177af908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:35.210762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0ca6d-69a9-4588-99b7-d3f6177af908 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:35.219153 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 233/447] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:35.220004 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0ca6d-69a9-4588-99b7-d3f6177af908 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:35.257712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:35.266986 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130cc30-ed6e-4648-a0e1-96f65207a440 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:35.299216 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba9778-c444-4755-9a15-682aa5f8ae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:35.300106 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba9778-c444-4755-9a15-682aa5f8ae44 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:35.300390 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba9778-c444-4755-9a15-682aa5f8ae44 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:35.300671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba9778-c444-4755-9a15-682aa5f8ae44 None admin] PATH_INFO: `/v3/projects/36b45cb287dd49428bc291b9834d0b1e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:35.301857 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba9778-c444-4755-9a15-682aa5f8ae44 None admin] RBAC: Authorizing `identity:get_project(project_id=36b45cb287dd49428bc291b9834d0b1e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:35.303436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba9778-c444-4755-9a15-682aa5f8ae44 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:35.305135 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 215/448] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:35 2022] GET /identity/v3/projects/36b45cb287dd49428bc291b9834d0b1e => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:35.306940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba9778-c444-4755-9a15-682aa5f8ae44 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:35.320941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0ca6d-69a9-4588-99b7-d3f6177af908 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:35.354445 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76a65-2af3-4f47-b55a-0e297544ce08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:35.355116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76a65-2af3-4f47-b55a-0e297544ce08 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:35.355402 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76a65-2af3-4f47-b55a-0e297544ce08 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:35.355689 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76a65-2af3-4f47-b55a-0e297544ce08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:35.356536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab76a65-2af3-4f47-b55a-0e297544ce08 None admin] RBAC: Adding query filter params (group.id=d1cb6cdec2244e5bb303a3b659d83293, role.id=b10cd0627cc84a9c92dba333901e60e1, scope.project.id=36b45cb287dd49428bc291b9834d0b1e) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:35.356890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab76a65-2af3-4f47-b55a-0e297544ce08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:35.358370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab76a65-2af3-4f47-b55a-0e297544ce08 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:35.364345 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 234/449] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jul 5 09:59:35 2022] GET /identity/v3/role_assignments?group.id=d1cb6cdec2244e5bb303a3b659d83293&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=b10cd0627cc84a9c92dba333901e60e1 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:35.365028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76a65-2af3-4f47-b55a-0e297544ce08 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:35.385777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:35.419886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba9778-c444-4755-9a15-682aa5f8ae44 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:35.456813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f451cc1-aeb5-4868-87e8-4b2f084e6d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:35.457480 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f451cc1-aeb5-4868-87e8-4b2f084e6d28 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:35.458063 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f451cc1-aeb5-4868-87e8-4b2f084e6d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:35.458423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f451cc1-aeb5-4868-87e8-4b2f084e6d28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:35.480738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76a65-2af3-4f47-b55a-0e297544ce08 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:35.490504 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:35.517367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28cca8-c1af-4d96-a0c4-280418c31517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:35.518267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28cca8-c1af-4d96-a0c4-280418c31517 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:35.518545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28cca8-c1af-4d96-a0c4-280418c31517 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:35.518823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28cca8-c1af-4d96-a0c4-280418c31517 None admin] PATH_INFO: `/v3/groups/d1cb6cdec2244e5bb303a3b659d83293` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:35.520084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28cca8-c1af-4d96-a0c4-280418c31517 None admin] RBAC: Authorizing `identity:get_group(group_id=d1cb6cdec2244e5bb303a3b659d83293)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:35.521614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28cca8-c1af-4d96-a0c4-280418c31517 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:35.523275 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 235/450] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:35 2022] GET /identity/v3/groups/d1cb6cdec2244e5bb303a3b659d83293 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:35.529045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28cca8-c1af-4d96-a0c4-280418c31517 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:35.543593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f451cc1-aeb5-4868-87e8-4b2f084e6d28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:35.545188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f451cc1-aeb5-4868-87e8-4b2f084e6d28 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:35.624364 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:35.640703 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28cca8-c1af-4d96-a0c4-280418c31517 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:35.656416 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 216/451] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:35.663049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f451cc1-aeb5-4868-87e8-4b2f084e6d28 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:35.679798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d350e-3bac-4143-9966-74ca3d8bd2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:35.681546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d350e-3bac-4143-9966-74ca3d8bd2ff None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:35.681546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d350e-3bac-4143-9966-74ca3d8bd2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:35.681546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d350e-3bac-4143-9966-74ca3d8bd2ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:35.681779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d350e-3bac-4143-9966-74ca3d8bd2ff None admin] RBAC: Adding query filter params (group.id=d1cb6cdec2244e5bb303a3b659d83293, role.id=70d38d0a968a4e9585c5873d24cfbbb1, scope.project.id=36b45cb287dd49428bc291b9834d0b1e) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:35.682058 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d350e-3bac-4143-9966-74ca3d8bd2ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:35.683614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d350e-3bac-4143-9966-74ca3d8bd2ff None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:35.690143 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 236/452] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jul 5 09:59:35 2022] GET /identity/v3/role_assignments?group.id=d1cb6cdec2244e5bb303a3b659d83293&scope.project.id=36b45cb287dd49428bc291b9834d0b1e&role.id=70d38d0a968a4e9585c5873d24cfbbb1 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:35.743995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:35.789048 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f451cc1-aeb5-4868-87e8-4b2f084e6d28 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:35.823151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d51c7-c974-406f-804f-d1d53619ee00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:35.824024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d51c7-c974-406f-804f-d1d53619ee00 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:35.824335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d51c7-c974-406f-804f-d1d53619ee00 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:35.824607 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d51c7-c974-406f-804f-d1d53619ee00 None admin] PATH_INFO: `/v3/projects/95874cd0621e46bbbfb452bfbd474389` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:35.825863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d51c7-c974-406f-804f-d1d53619ee00 None admin] RBAC: Authorizing `identity:get_project(project_id=95874cd0621e46bbbfb452bfbd474389)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:35.827441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d51c7-c974-406f-804f-d1d53619ee00 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:35.829149 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 217/453] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 09:59:35 2022] GET /identity/v3/projects/95874cd0621e46bbbfb452bfbd474389 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:35.835247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d350e-3bac-4143-9966-74ca3d8bd2ff None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:35.944917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d350e-3bac-4143-9966-74ca3d8bd2ff None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:35.978736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b239e-6aaa-44f7-b769-0ac66bf50021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:35.979585 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b239e-6aaa-44f7-b769-0ac66bf50021 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:35.979887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b239e-6aaa-44f7-b769-0ac66bf50021 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:35.980172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b239e-6aaa-44f7-b769-0ac66bf50021 None admin] PATH_INFO: `/v3/groups/3780647aeafc4e17b1eb24faa4b8de66` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:35.981342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b239e-6aaa-44f7-b769-0ac66bf50021 None admin] RBAC: Authorizing `identity:get_group(group_id=3780647aeafc4e17b1eb24faa4b8de66)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:35.982819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b239e-6aaa-44f7-b769-0ac66bf50021 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:35.984442 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 237/454] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jul 5 09:59:35 2022] GET /identity/v3/groups/3780647aeafc4e17b1eb24faa4b8de66 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:35.990697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d51c7-c974-406f-804f-d1d53619ee00 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:36.090453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d51c7-c974-406f-804f-d1d53619ee00 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:36.116495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:36.125482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30afd8-141a-4fa3-abc7-e9b820428ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:36.126131 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30afd8-141a-4fa3-abc7-e9b820428ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:36.126519 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30afd8-141a-4fa3-abc7-e9b820428ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:36.126792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30afd8-141a-4fa3-abc7-e9b820428ce2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:36.127613 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30afd8-141a-4fa3-abc7-e9b820428ce2 None admin] RBAC: Adding query filter params (group.id=3780647aeafc4e17b1eb24faa4b8de66, role.id=c184a177010144429f495f3290d28221, scope.project.id=95874cd0621e46bbbfb452bfbd474389) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:36.127920 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30afd8-141a-4fa3-abc7-e9b820428ce2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:36.129331 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30afd8-141a-4fa3-abc7-e9b820428ce2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:36.135347 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 218/455] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jul 5 09:59:35 2022] GET /identity/v3/role_assignments?group.id=3780647aeafc4e17b1eb24faa4b8de66&scope.project.id=95874cd0621e46bbbfb452bfbd474389&role.id=c184a177010144429f495f3290d28221 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:38.644364 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec94016-433a-4c04-b9c6-34f265053789 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:38.645399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec94016-433a-4c04-b9c6-34f265053789 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:38.645919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec94016-433a-4c04-b9c6-34f265053789 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:38.647810 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 238/456] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:38.653013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d1d96-f7fb-40d3-8572-b47edc905ef8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:38.653815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d1d96-f7fb-40d3-8572-b47edc905ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:38.654357 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d1d96-f7fb-40d3-8572-b47edc905ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:38.674534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1c5d1d96-f7fb-40d3-8572-b47edc905ef8 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:38.674534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1c5d1d96-f7fb-40d3-8572-b47edc905ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:38.676702 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac4989-e8d1-4364-bbfc-00f69254edbe None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:38.677269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac4989-e8d1-4364-bbfc-00f69254edbe None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:38.678747 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac4989-e8d1-4364-bbfc-00f69254edbe None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:38.680835 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 239/457] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:38.693495 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee896dc8-4d21-4db4-8afc-f77b61d8b061 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:38.694247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee896dc8-4d21-4db4-8afc-f77b61d8b061 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee896dc8-4d21-4db4-8afc-f77b61d8b061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:38.720066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ee896dc8-4d21-4db4-8afc-f77b61d8b061 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:38.721242 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ee896dc8-4d21-4db4-8afc-f77b61d8b061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:38.752010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:38.766006 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 219/458] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:38.778025 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fc1c8-716e-4010-9134-a5f8107a4c42 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:38.778345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fc1c8-716e-4010-9134-a5f8107a4c42 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:38.778625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fc1c8-716e-4010-9134-a5f8107a4c42 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:38.780201 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 220/459] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:38.785496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ff24f-fc7e-4e08-8196-324a9d42fc4d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:38.785784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ff24f-fc7e-4e08-8196-324a9d42fc4d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:38.786058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ff24f-fc7e-4e08-8196-324a9d42fc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:38.805894 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-969ff24f-fc7e-4e08-8196-324a9d42fc4d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:38.807251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-969ff24f-fc7e-4e08-8196-324a9d42fc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:38.831542 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 240/460] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:38.833431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee896dc8-4d21-4db4-8afc-f77b61d8b061 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:38.894551 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 221/461] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:38.895376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ff24f-fc7e-4e08-8196-324a9d42fc4d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:38.946895 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee896dc8-4d21-4db4-8afc-f77b61d8b061 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:38.982271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebfa3c-284c-4e12-ae11-10c0be5e36c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:38.983084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebfa3c-284c-4e12-ae11-10c0be5e36c2 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:38.983361 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebfa3c-284c-4e12-ae11-10c0be5e36c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:38.983630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebfa3c-284c-4e12-ae11-10c0be5e36c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:38.994155 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ebfa3c-284c-4e12-ae11-10c0be5e36c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:38.995647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ebfa3c-284c-4e12-ae11-10c0be5e36c2 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:39.004752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:39.014626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-27ebfa3c-284c-4e12-ae11-10c0be5e36c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:39.016826 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:39.017670 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:39.019064 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:39.019064 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:39.019064 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:39.019064 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:39.019064 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 241/462] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:39.019333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebfa3c-284c-4e12-ae11-10c0be5e36c2 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:39.030956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ff24f-fc7e-4e08-8196-324a9d42fc4d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:39.073190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1044a35-2403-4f3a-934d-126b5ccd31a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:39.074080 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044a35-2403-4f3a-934d-126b5ccd31a6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:39.074393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044a35-2403-4f3a-934d-126b5ccd31a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:39.074669 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044a35-2403-4f3a-934d-126b5ccd31a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:39.086194 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1044a35-2403-4f3a-934d-126b5ccd31a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:39.087730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1044a35-2403-4f3a-934d-126b5ccd31a6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d1044a35-2403-4f3a-934d-126b5ccd31a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:39.108934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:39.109899 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:39.110866 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:39.110866 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:39.110866 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:39.110866 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:39.110866 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 222/463] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:39.111921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1044a35-2403-4f3a-934d-126b5ccd31a6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:39.150103 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebfa3c-284c-4e12-ae11-10c0be5e36c2 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:39.186341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf1dc1-2e4c-4507-bea0-784b222d44e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:39.187146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf1dc1-2e4c-4507-bea0-784b222d44e1 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:39.187426 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf1dc1-2e4c-4507-bea0-784b222d44e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:39.187700 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf1dc1-2e4c-4507-bea0-784b222d44e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:39.198088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf1dc1-2e4c-4507-bea0-784b222d44e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:39.199634 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf1dc1-2e4c-4507-bea0-784b222d44e1 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-bfaf1dc1-2e4c-4507-bea0-784b222d44e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:39.217621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:39.219660 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:39.221828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:39.221828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:39.221828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:39.221828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:39.221828 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 242/464] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:39.221828 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf1dc1-2e4c-4507-bea0-784b222d44e1 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:39.248543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1044a35-2403-4f3a-934d-126b5ccd31a6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:39.252007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:39.279194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:39.295471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b2a80-38da-40d9-bcdc-855d9e513d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:39.296148 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b2a80-38da-40d9-bcdc-855d9e513d86 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:39.296432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b2a80-38da-40d9-bcdc-855d9e513d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:39.296708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b2a80-38da-40d9-bcdc-855d9e513d86 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:39.297441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b2a80-38da-40d9-bcdc-855d9e513d86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:39.297773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b2a80-38da-40d9-bcdc-855d9e513d86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:39.299184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b2a80-38da-40d9-bcdc-855d9e513d86 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:39.314501 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 223/465] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:39.314501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b2a80-38da-40d9-bcdc-855d9e513d86 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:39.331546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf1dc1-2e4c-4507-bea0-784b222d44e1 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:39.368212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de318b-3481-4ac5-bbd7-b556f0180ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:39.368869 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de318b-3481-4ac5-bbd7-b556f0180ade None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:39.369148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de318b-3481-4ac5-bbd7-b556f0180ade None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:39.369420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de318b-3481-4ac5-bbd7-b556f0180ade None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:39.370132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de318b-3481-4ac5-bbd7-b556f0180ade None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:39.370477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de318b-3481-4ac5-bbd7-b556f0180ade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:39.371863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de318b-3481-4ac5-bbd7-b556f0180ade None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:39.387391 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 243/466] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:39.387391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151e169-50ba-4402-8137-674c73a20c79 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:39.387391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151e169-50ba-4402-8137-674c73a20c79 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:39.387391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151e169-50ba-4402-8137-674c73a20c79 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:39.390380 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 244/467] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:39.390887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151e169-50ba-4402-8137-674c73a20c79 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:39.414464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b2a80-38da-40d9-bcdc-855d9e513d86 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:39.450605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01960b5-3f23-4d75-b1d1-e5cf6b45b6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:39.451277 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01960b5-3f23-4d75-b1d1-e5cf6b45b6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:39.451564 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01960b5-3f23-4d75-b1d1-e5cf6b45b6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:39.451970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01960b5-3f23-4d75-b1d1-e5cf6b45b6a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:39.452573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01960b5-3f23-4d75-b1d1-e5cf6b45b6a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:39.452873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01960b5-3f23-4d75-b1d1-e5cf6b45b6a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:39.454293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01960b5-3f23-4d75-b1d1-e5cf6b45b6a8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:39.470266 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 224/468] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:39.470266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e504-c0ab-4bc8-9a2a-3c4ab52dd436 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:39.470266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e504-c0ab-4bc8-9a2a-3c4ab52dd436 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:39.470266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e504-c0ab-4bc8-9a2a-3c4ab52dd436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:39.485446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6b89e504-c0ab-4bc8-9a2a-3c4ab52dd436 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:39.486662 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6b89e504-c0ab-4bc8-9a2a-3c4ab52dd436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:39.498491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151e169-50ba-4402-8137-674c73a20c79 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:39.506125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:39.512175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:39.536107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bc518-9999-45fa-b9f9-ba6be1cefc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:39.536845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bc518-9999-45fa-b9f9-ba6be1cefc7a None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:39.537124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bc518-9999-45fa-b9f9-ba6be1cefc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:39.537398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bc518-9999-45fa-b9f9-ba6be1cefc7a None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:39.538118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bc518-9999-45fa-b9f9-ba6be1cefc7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:39.539591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bc518-9999-45fa-b9f9-ba6be1cefc7a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:39.558786 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 225/469] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:39.559885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89e504-c0ab-4bc8-9a2a-3c4ab52dd436 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:39.578259 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 245/470] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:59:39 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:59:39.578259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bc518-9999-45fa-b9f9-ba6be1cefc7a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:39.690575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89e504-c0ab-4bc8-9a2a-3c4ab52dd436 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:39.703301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bc518-9999-45fa-b9f9-ba6be1cefc7a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:39.727607 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ab66f-9bfa-4293-a788-1ea02606342f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:39.728477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ab66f-9bfa-4293-a788-1ea02606342f None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:39.728721 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ab66f-9bfa-4293-a788-1ea02606342f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:39.729065 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ab66f-9bfa-4293-a788-1ea02606342f None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:39.729850 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ab66f-9bfa-4293-a788-1ea02606342f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:39.731646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ab66f-9bfa-4293-a788-1ea02606342f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:39.744897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63beadcb-a9d1-4a3d-a340-0c8dd126eb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:39.746005 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63beadcb-a9d1-4a3d-a340-0c8dd126eb53 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:39.746442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63beadcb-a9d1-4a3d-a340-0c8dd126eb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:39.746990 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63beadcb-a9d1-4a3d-a340-0c8dd126eb53 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:39.747563 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63beadcb-a9d1-4a3d-a340-0c8dd126eb53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:39.750679 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63beadcb-a9d1-4a3d-a340-0c8dd126eb53 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:39.762607 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 226/471] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:59:39 2022] POST /identity/v3/users => generated 248 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:59:39.763715 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ab66f-9bfa-4293-a788-1ea02606342f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:39.781535 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 246/472] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:59:39 2022] POST /identity/v3/users => generated 248 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:59:39.784198 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:39.897918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ab66f-9bfa-4293-a788-1ea02606342f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:39.938120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2e2bf-5266-420d-be21-3c4fd09cdb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:39.939393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2e2bf-5266-420d-be21-3c4fd09cdb47 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:39.939889 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2e2bf-5266-420d-be21-3c4fd09cdb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:39.940417 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2e2bf-5266-420d-be21-3c4fd09cdb47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:39.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2e2bf-5266-420d-be21-3c4fd09cdb47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:39.956755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2e2bf-5266-420d-be21-3c4fd09cdb47 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d9b2e2bf-5266-420d-be21-3c4fd09cdb47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:39.977671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:39.978619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:39.979449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:39.979449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:39.979449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:39.979449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:39.982355 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 227/473] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:39.987427 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63beadcb-a9d1-4a3d-a340-0c8dd126eb53 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:40.018527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:40.062096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89028298-a1f6-45d6-9db9-8208f7121468 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:40.062526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89028298-a1f6-45d6-9db9-8208f7121468 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:40.062730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89028298-a1f6-45d6-9db9-8208f7121468 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:40.064105 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 228/474] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 09:59:40.069716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d86e2-d40e-488d-989e-0462444adf09 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:40.070876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d86e2-d40e-488d-989e-0462444adf09 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:40.070876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d86e2-d40e-488d-989e-0462444adf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:40.094827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ef6d86e2-d40e-488d-989e-0462444adf09 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:40.096031 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ef6d86e2-d40e-488d-989e-0462444adf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:40.105042 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63beadcb-a9d1-4a3d-a340-0c8dd126eb53 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:40.145058 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde55ad0-1926-4826-821c-c4ecb2d495d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:40.146249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde55ad0-1926-4826-821c-c4ecb2d495d7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:40.146554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde55ad0-1926-4826-821c-c4ecb2d495d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:40.147222 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde55ad0-1926-4826-821c-c4ecb2d495d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:40.148307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde55ad0-1926-4826-821c-c4ecb2d495d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:40.148617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde55ad0-1926-4826-821c-c4ecb2d495d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:40.150698 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde55ad0-1926-4826-821c-c4ecb2d495d7 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:40.160721 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:40.168521 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 247/475] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:40.174359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde55ad0-1926-4826-821c-c4ecb2d495d7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:40.174606 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 229/476] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:40.197583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d86e2-d40e-488d-989e-0462444adf09 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:40.283317 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:40.301261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde55ad0-1926-4826-821c-c4ecb2d495d7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:40.360266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c85dac-5808-4e73-b8d8-508b04d1d150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:40.360978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c85dac-5808-4e73-b8d8-508b04d1d150 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:40.361261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c85dac-5808-4e73-b8d8-508b04d1d150 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:40.361537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c85dac-5808-4e73-b8d8-508b04d1d150 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:40.362269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c85dac-5808-4e73-b8d8-508b04d1d150 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:40.363838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c85dac-5808-4e73-b8d8-508b04d1d150 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:40.367190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d86e2-d40e-488d-989e-0462444adf09 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:40.385748 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 248/477] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:59:40 2022] POST /identity/v3/users => generated 249 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 09:59:40.424376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677e3d5-9f06-476d-b694-1ccf4ca1a465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:40.425386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677e3d5-9f06-476d-b694-1ccf4ca1a465 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:40.425843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677e3d5-9f06-476d-b694-1ccf4ca1a465 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:40.426313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677e3d5-9f06-476d-b694-1ccf4ca1a465 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:40.427488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677e3d5-9f06-476d-b694-1ccf4ca1a465 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:40.429103 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677e3d5-9f06-476d-b694-1ccf4ca1a465 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:40.444793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.sql.core [None req-5677e3d5-9f06-476d-b694-1ccf4ca1a465 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 05 09:59:40.444793 dv-cinder-master-3-smb devstack@keystone.service[80377]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 05 09:59:40.444793 dv-cinder-master-3-smb devstack@keystone.service[80377]: [parameters: {'id': 'b646140e30e04b62b080900682ceb2df', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 05 09:59:40.444793 dv-cinder-master-3-smb devstack@keystone.service[80377]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80377) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5677e3d5-9f06-476d-b694-1ccf4ca1a465 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application conn.query(q) Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application result.read() Jul 05 09:59:40.466256 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return ref.to_dict() Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 09:59:40.467206 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application next(self.gen) Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application yield resource Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application next(self.gen) Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.session.rollback() Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.force_reraise() Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise self.value Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application session.commit() Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 05 09:59:40.468091 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._prepare_impl() Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.session.flush() Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._flush(objects) Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application compat.raise_( Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application flush_context.execute() Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rec.execute(self) Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 05 09:59:40.468926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 09:59:40.469840 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application conn.query(q) Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application result.read() Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 09:59:40.470681 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [parameters: {'id': 'b646140e30e04b62b080900682ceb2df', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:40.471458 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 05 09:59:40.477117 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:40.478193 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 230/478] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Jul 5 09:59:40 2022] POST /identity/v3/roles => generated 144 bytes in 275 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:40.478496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c85dac-5808-4e73-b8d8-508b04d1d150 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:40.602617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c85dac-5808-4e73-b8d8-508b04d1d150 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:40.637416 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a571-5f5c-4196-8a83-2adf2ad6149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:40.638307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a571-5f5c-4196-8a83-2adf2ad6149d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:40.638587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a571-5f5c-4196-8a83-2adf2ad6149d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:40.638899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a571-5f5c-4196-8a83-2adf2ad6149d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:40.646930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:40.651205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296a571-5f5c-4196-8a83-2adf2ad6149d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-b296a571-5f5c-4196-8a83-2adf2ad6149d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:40.660331 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:40.662757 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:40.663813 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 249/479] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 09:59:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:40.668520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677e3d5-9f06-476d-b694-1ccf4ca1a465 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:40.736902 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:40.783027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677e3d5-9f06-476d-b694-1ccf4ca1a465 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:40.817002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a997d5-da96-4ddd-8649-0d5f8e61cdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:40.817743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a997d5-da96-4ddd-8649-0d5f8e61cdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:40.818031 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a997d5-da96-4ddd-8649-0d5f8e61cdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:40.818338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a997d5-da96-4ddd-8649-0d5f8e61cdd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:40.819103 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a997d5-da96-4ddd-8649-0d5f8e61cdd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:40.819411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a997d5-da96-4ddd-8649-0d5f8e61cdd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:40.820790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a997d5-da96-4ddd-8649-0d5f8e61cdd4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:40.829590 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 231/480] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:42.151019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f53ea-bb11-4ed0-8257-f6a028e34d49 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:42.151729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f53ea-bb11-4ed0-8257-f6a028e34d49 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:42.151729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f53ea-bb11-4ed0-8257-f6a028e34d49 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:42.153087 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 250/481] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:42.157965 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a06d0e-bcb2-4df5-930f-eec361bb8cdc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:42.158516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a06d0e-bcb2-4df5-930f-eec361bb8cdc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:42.158671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a06d0e-bcb2-4df5-930f-eec361bb8cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:42.179558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e3a06d0e-bcb2-4df5-930f-eec361bb8cdc None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:42.181098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e3a06d0e-bcb2-4df5-930f-eec361bb8cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:42.275903 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 232/482] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:42.299598 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f53ea-bb11-4ed0-8257-f6a028e34d49 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:42.380575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b226bb-16aa-4819-8051-f735ee2a90f6 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:42.381512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b226bb-16aa-4819-8051-f735ee2a90f6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:42.382313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b226bb-16aa-4819-8051-f735ee2a90f6 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:42.388037 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 233/483] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 09:59:42.395777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4c910-c7b6-407d-b7e4-33f9283b77e4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:42.396445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4c910-c7b6-407d-b7e4-33f9283b77e4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:42.397108 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4c910-c7b6-407d-b7e4-33f9283b77e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:42.425504 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f4f4c910-c7b6-407d-b7e4-33f9283b77e4 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:42.427229 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f4f4c910-c7b6-407d-b7e4-33f9283b77e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:42.435286 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f53ea-bb11-4ed0-8257-f6a028e34d49 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:42.470768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928188fb-4b0f-4507-b14c-9bf4367e3ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:42.471569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928188fb-4b0f-4507-b14c-9bf4367e3ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:42.471857 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928188fb-4b0f-4507-b14c-9bf4367e3ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:42.472127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928188fb-4b0f-4507-b14c-9bf4367e3ab4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:42.482820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928188fb-4b0f-4507-b14c-9bf4367e3ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:42.487595 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-928188fb-4b0f-4507-b14c-9bf4367e3ab4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:42.492205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:42.493321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:42.494330 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:42.497378 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 251/484] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:42.498849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43882116-1388-4caf-9267-88ec6be76ec6 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:42.499389 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43882116-1388-4caf-9267-88ec6be76ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:42.499796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43882116-1388-4caf-9267-88ec6be76ec6 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:42.501503 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 252/485] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:42.507360 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17354e-7555-464d-a48f-2470f261131e None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:42.507645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17354e-7555-464d-a48f-2470f261131e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:42.507925 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17354e-7555-464d-a48f-2470f261131e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:42.525668 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 234/486] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:42.526697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ab17354e-7555-464d-a48f-2470f261131e None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:42.528086 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ab17354e-7555-464d-a48f-2470f261131e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:42.535589 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4c910-c7b6-407d-b7e4-33f9283b77e4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:42.585039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:42.599234 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 253/487] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:42.606461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17354e-7555-464d-a48f-2470f261131e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:42.708453 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17354e-7555-464d-a48f-2470f261131e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:42.712287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4c910-c7b6-407d-b7e4-33f9283b77e4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:42.744436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab52ff-c5e5-4a09-bf24-54489fb96106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:42.745705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab52ff-c5e5-4a09-bf24-54489fb96106 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:42.746244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab52ff-c5e5-4a09-bf24-54489fb96106 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:42.746710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab52ff-c5e5-4a09-bf24-54489fb96106 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:42.755333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccc17c-7918-4f5f-9447-983c62772a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:42.756337 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccc17c-7918-4f5f-9447-983c62772a85 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:42.756940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccc17c-7918-4f5f-9447-983c62772a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:42.757487 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccc17c-7918-4f5f-9447-983c62772a85 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:42.758464 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab52ff-c5e5-4a09-bf24-54489fb96106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:42.759116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cccc17c-7918-4f5f-9447-983c62772a85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:42.759861 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cccc17c-7918-4f5f-9447-983c62772a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:42.761889 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cccc17c-7918-4f5f-9447-983c62772a85 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-c2ab52ff-c5e5-4a09-bf24-54489fb96106 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:42.769583 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:42.771030 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 254/488] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:42.771993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab52ff-c5e5-4a09-bf24-54489fb96106 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:42.779508 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 235/489] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:42.781361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccc17c-7918-4f5f-9447-983c62772a85 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:42.831008 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:42.839828 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:42.888737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab52ff-c5e5-4a09-bf24-54489fb96106 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:42.910814 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccc17c-7918-4f5f-9447-983c62772a85 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:42.924782 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba3caf-a59d-415f-806b-7acaa56513af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:42.925834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba3caf-a59d-415f-806b-7acaa56513af None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:42.926287 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba3caf-a59d-415f-806b-7acaa56513af None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:42.926697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba3caf-a59d-415f-806b-7acaa56513af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:42.938026 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba3caf-a59d-415f-806b-7acaa56513af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-92ba3caf-a59d-415f-806b-7acaa56513af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:42.946568 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:42.947471 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:42.948260 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:42.951108 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 255/490] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:42.952138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba3caf-a59d-415f-806b-7acaa56513af None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:42.963180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958f63e-3195-467a-a8ea-638ea677fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:42.964196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958f63e-3195-467a-a8ea-638ea677fd02 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:42.964552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958f63e-3195-467a-a8ea-638ea677fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:42.965005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958f63e-3195-467a-a8ea-638ea677fd02 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:42.966215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958f63e-3195-467a-a8ea-638ea677fd02 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:42.966746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958f63e-3195-467a-a8ea-638ea677fd02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:42.968825 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958f63e-3195-467a-a8ea-638ea677fd02 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:42.980637 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 236/491] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:42.981995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958f63e-3195-467a-a8ea-638ea677fd02 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:43.066372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba3caf-a59d-415f-806b-7acaa56513af None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:43.077860 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:43.105353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c29712-d20a-432a-ad78-eafce56ea232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:43.106657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c29712-d20a-432a-ad78-eafce56ea232 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.107172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c29712-d20a-432a-ad78-eafce56ea232 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.107681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c29712-d20a-432a-ad78-eafce56ea232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.112445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958f63e-3195-467a-a8ea-638ea677fd02 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:43.114301 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:43.121694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c29712-d20a-432a-ad78-eafce56ea232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:43.123694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c29712-d20a-432a-ad78-eafce56ea232 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-94c29712-d20a-432a-ad78-eafce56ea232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:43.141992 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:43.142910 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:43.143748 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:43.143748 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:43.143748 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:43.143748 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:43.146113 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 256/492] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:43.147390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c29712-d20a-432a-ad78-eafce56ea232 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:43.157368 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac905274-7f9f-47cf-bcef-6076b92fde9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:43.158576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac905274-7f9f-47cf-bcef-6076b92fde9d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.159793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac905274-7f9f-47cf-bcef-6076b92fde9d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.160361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac905274-7f9f-47cf-bcef-6076b92fde9d None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.161734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac905274-7f9f-47cf-bcef-6076b92fde9d None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:43.162211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac905274-7f9f-47cf-bcef-6076b92fde9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:43.164040 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac905274-7f9f-47cf-bcef-6076b92fde9d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:43.180867 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 237/493] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:43.182173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac905274-7f9f-47cf-bcef-6076b92fde9d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:43.253144 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c29712-d20a-432a-ad78-eafce56ea232 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:43.287738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343907a-c72a-41f0-9ba6-5c4861b0cb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:43.288544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343907a-c72a-41f0-9ba6-5c4861b0cb02 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.288870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343907a-c72a-41f0-9ba6-5c4861b0cb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.289129 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343907a-c72a-41f0-9ba6-5c4861b0cb02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.299938 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343907a-c72a-41f0-9ba6-5c4861b0cb02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:43.301477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343907a-c72a-41f0-9ba6-5c4861b0cb02 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:43.311172 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac905274-7f9f-47cf-bcef-6076b92fde9d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-8343907a-c72a-41f0-9ba6-5c4861b0cb02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:43.318828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:43.319695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:43.320551 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:43.320551 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:43.320551 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:43.320551 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:43.320551 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 257/494] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:43.320832 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343907a-c72a-41f0-9ba6-5c4861b0cb02 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:43.343918 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:43.349537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955efe47-5445-4fd9-b87c-f68785087452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:43.350262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955efe47-5445-4fd9-b87c-f68785087452 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.350931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955efe47-5445-4fd9-b87c-f68785087452 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.350931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955efe47-5445-4fd9-b87c-f68785087452 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.351514 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955efe47-5445-4fd9-b87c-f68785087452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:43.351807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955efe47-5445-4fd9-b87c-f68785087452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:43.353177 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955efe47-5445-4fd9-b87c-f68785087452 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:43.364951 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 238/495] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:43.366287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6dccd-244a-416a-8b25-fef6ecffb6d4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.366661 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6dccd-244a-416a-8b25-fef6ecffb6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.366905 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6dccd-244a-416a-8b25-fef6ecffb6d4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.372219 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 239/496] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:43.373057 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6dccd-244a-416a-8b25-fef6ecffb6d4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:43.396912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:43.428254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343907a-c72a-41f0-9ba6-5c4861b0cb02 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:43.460124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70311a84-55f7-4470-9983-8ec7e4360bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:43.460916 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70311a84-55f7-4470-9983-8ec7e4360bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.461199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70311a84-55f7-4470-9983-8ec7e4360bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.461469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70311a84-55f7-4470-9983-8ec7e4360bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.470693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6dccd-244a-416a-8b25-fef6ecffb6d4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:43.471773 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70311a84-55f7-4470-9983-8ec7e4360bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:43.473237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70311a84-55f7-4470-9983-8ec7e4360bd2 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-70311a84-55f7-4470-9983-8ec7e4360bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:43.490022 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:43.491054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:43.492096 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:43.492096 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:43.492096 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:43.492096 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:43.492096 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 258/497] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:43.492737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d0c87-3ed6-4166-b746-44517d7d4f0e None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.493019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d0c87-3ed6-4166-b746-44517d7d4f0e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.493289 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d0c87-3ed6-4166-b746-44517d7d4f0e None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.498815 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 259/498] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:43.499600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d0c87-3ed6-4166-b746-44517d7d4f0e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:43.507940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262092c-05ac-4793-9962-eb3fd32abcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:43.508550 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262092c-05ac-4793-9962-eb3fd32abcdd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.508834 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262092c-05ac-4793-9962-eb3fd32abcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.509508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262092c-05ac-4793-9962-eb3fd32abcdd None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.510272 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262092c-05ac-4793-9962-eb3fd32abcdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:43.510586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262092c-05ac-4793-9962-eb3fd32abcdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:43.512351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262092c-05ac-4793-9962-eb3fd32abcdd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:43.524018 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 240/499] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:43.525323 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44f185-ebbc-438b-9b1e-8c59cda1c5eb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.525598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44f185-ebbc-438b-9b1e-8c59cda1c5eb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.525831 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44f185-ebbc-438b-9b1e-8c59cda1c5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.541952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ec44f185-ebbc-438b-9b1e-8c59cda1c5eb None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:43.543214 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ec44f185-ebbc-438b-9b1e-8c59cda1c5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:43.563961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:43.598786 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d0c87-3ed6-4166-b746-44517d7d4f0e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:43.611171 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 241/500] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:43.612050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44f185-ebbc-438b-9b1e-8c59cda1c5eb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:43.633772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:43.634080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd780d-2540-4895-8380-f5003404f5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:43.634991 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd780d-2540-4895-8380-f5003404f5bc None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.635266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd780d-2540-4895-8380-f5003404f5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.635543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd780d-2540-4895-8380-f5003404f5bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.643690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd780d-2540-4895-8380-f5003404f5bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:43.645142 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd780d-2540-4895-8380-f5003404f5bc None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-30dd780d-2540-4895-8380-f5003404f5bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:43.660363 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:43.661320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 260/501] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jul 5 09:59:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54becb70-e79c-4d54-b71e-a0b2918d4b66 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.666916 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54becb70-e79c-4d54-b71e-a0b2918d4b66 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.667653 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54becb70-e79c-4d54-b71e-a0b2918d4b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.679621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-54becb70-e79c-4d54-b71e-a0b2918d4b66 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:43.680817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-54becb70-e79c-4d54-b71e-a0b2918d4b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:43.718203 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44f185-ebbc-438b-9b1e-8c59cda1c5eb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:43.744579 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 261/502] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:43.745393 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54becb70-e79c-4d54-b71e-a0b2918d4b66 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:43.751125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a42da6-d6cf-4250-86f3-9824b81b0a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:43.751792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a42da6-d6cf-4250-86f3-9824b81b0a63 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.752075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a42da6-d6cf-4250-86f3-9824b81b0a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.752349 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a42da6-d6cf-4250-86f3-9824b81b0a63 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.753109 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a42da6-d6cf-4250-86f3-9824b81b0a63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:43.753418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a42da6-d6cf-4250-86f3-9824b81b0a63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:43.755767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a42da6-d6cf-4250-86f3-9824b81b0a63 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:43.770092 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 242/503] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:43.770092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a42da6-d6cf-4250-86f3-9824b81b0a63 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:43.814309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:43.844604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:43.859484 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54becb70-e79c-4d54-b71e-a0b2918d4b66 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:43.868424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a42da6-d6cf-4250-86f3-9824b81b0a63 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:43.891808 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358ad2f-bdde-421a-ad4c-cfdd25d1c5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:43.892611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358ad2f-bdde-421a-ad4c-cfdd25d1c5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.892890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358ad2f-bdde-421a-ad4c-cfdd25d1c5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.893158 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358ad2f-bdde-421a-ad4c-cfdd25d1c5d0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.900067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be37cc-6238-4cbf-9d2a-33243e8f1aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:43.900206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358ad2f-bdde-421a-ad4c-cfdd25d1c5d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:43.900856 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be37cc-6238-4cbf-9d2a-33243e8f1aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:43.901133 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be37cc-6238-4cbf-9d2a-33243e8f1aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:43.901398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be37cc-6238-4cbf-9d2a-33243e8f1aa9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:43.901618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358ad2f-bdde-421a-ad4c-cfdd25d1c5d0 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:43.911273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be37cc-6238-4cbf-9d2a-33243e8f1aa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-2358ad2f-bdde-421a-ad4c-cfdd25d1c5d0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:43.916173 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:43.917117 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:43.918024 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:43.918024 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:43.918024 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:43.918024 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:43.918024 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:43.918024 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:43.918024 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:43.918024 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:43.918024 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 05 09:59:43.918024 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:43.918024 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 262/504] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:43.918630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358ad2f-bdde-421a-ad4c-cfdd25d1c5d0 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-63be37cc-6238-4cbf-9d2a-33243e8f1aa9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:43.919254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:43.920079 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:43.920873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:43.922429 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 243/505] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:43.923088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be37cc-6238-4cbf-9d2a-33243e8f1aa9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:44.016246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358ad2f-bdde-421a-ad4c-cfdd25d1c5d0 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:44.021602 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be37cc-6238-4cbf-9d2a-33243e8f1aa9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:44.032803 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:44.050422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1d346-7240-425a-afe3-fc9098d4afb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:44.051087 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1d346-7240-425a-afe3-fc9098d4afb7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:44.051370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1d346-7240-425a-afe3-fc9098d4afb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:44.051637 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1d346-7240-425a-afe3-fc9098d4afb7 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:44.052562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1d346-7240-425a-afe3-fc9098d4afb7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:44.052866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1d346-7240-425a-afe3-fc9098d4afb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:44.054330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1d346-7240-425a-afe3-fc9098d4afb7 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:44.055700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d9a4a-57ba-4b21-bba2-c724bd6d134e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:44.056536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d9a4a-57ba-4b21-bba2-c724bd6d134e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:44.056816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d9a4a-57ba-4b21-bba2-c724bd6d134e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:44.057089 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d9a4a-57ba-4b21-bba2-c724bd6d134e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:44.059157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:44.069579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d9a4a-57ba-4b21-bba2-c724bd6d134e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:44.071078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d9a4a-57ba-4b21-bba2-c724bd6d134e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c91d9a4a-57ba-4b21-bba2-c724bd6d134e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:44.089088 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:44.090101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:44.091013 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:44.091013 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:44.091013 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:44.091013 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:44.091013 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 244/506] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:44.091278 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 263/507] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jul 5 09:59:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:44.091682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1d346-7240-425a-afe3-fc9098d4afb7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:44.092631 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d9a4a-57ba-4b21-bba2-c724bd6d134e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:44.189239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1d346-7240-425a-afe3-fc9098d4afb7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:44.189542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d9a4a-57ba-4b21-bba2-c724bd6d134e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:44.224000 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d3440-9be2-443b-80aa-16b409955b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:44.225089 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d3440-9be2-443b-80aa-16b409955b2c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:44.225089 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d3440-9be2-443b-80aa-16b409955b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:44.225286 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d3440-9be2-443b-80aa-16b409955b2c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:44.226538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3bbeb-b743-49ce-8368-c8fa57ba04ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:44.227296 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3bbeb-b743-49ce-8368-c8fa57ba04ed None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:44.227562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3bbeb-b743-49ce-8368-c8fa57ba04ed None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:44.227858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3bbeb-b743-49ce-8368-c8fa57ba04ed None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:44.228676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3bbeb-b743-49ce-8368-c8fa57ba04ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:44.228983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3bbeb-b743-49ce-8368-c8fa57ba04ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:44.230464 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3bbeb-b743-49ce-8368-c8fa57ba04ed None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:44.232849 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d3440-9be2-443b-80aa-16b409955b2c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:44.234308 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d3440-9be2-443b-80aa-16b409955b2c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:44.247340 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 264/508] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jul 5 09:59:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:44.249362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3bbeb-b743-49ce-8368-c8fa57ba04ed None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-341d3440-9be2-443b-80aa-16b409955b2c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:44.249962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:44.250997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:44.251885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:44.251885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:44.251885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:44.251885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:44.251885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:44.251885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:44.251885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:44.251885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:44.251885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 09:59:44.251885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:44.251885 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 245/509] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:44.252432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d3440-9be2-443b-80aa-16b409955b2c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:44.281473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:44.301593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:44.349428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3bbeb-b743-49ce-8368-c8fa57ba04ed None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:44.353498 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d3440-9be2-443b-80aa-16b409955b2c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:44.385833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87988a2-b824-4799-b0ba-f7307451c8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:44.386569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87988a2-b824-4799-b0ba-f7307451c8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:44.386849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87988a2-b824-4799-b0ba-f7307451c8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:44.387239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87988a2-b824-4799-b0ba-f7307451c8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:44.388014 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87988a2-b824-4799-b0ba-f7307451c8c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:44.388324 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87988a2-b824-4799-b0ba-f7307451c8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:44.389735 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87988a2-b824-4799-b0ba-f7307451c8c5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:44.393906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd67548-2b99-4392-aa33-4c3b4924322e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:44.394625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd67548-2b99-4392-aa33-4c3b4924322e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:44.394954 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd67548-2b99-4392-aa33-4c3b4924322e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:44.395233 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd67548-2b99-4392-aa33-4c3b4924322e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:44.398153 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 265/510] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:44.398985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87988a2-b824-4799-b0ba-f7307451c8c5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:44.467719 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd67548-2b99-4392-aa33-4c3b4924322e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:44.469639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd67548-2b99-4392-aa33-4c3b4924322e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:44.494826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:44.501936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87988a2-b824-4799-b0ba-f7307451c8c5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:44.515783 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:44.536269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fa121-8e88-4324-bc80-8698e7aceda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:44.536872 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fa121-8e88-4324-bc80-8698e7aceda6 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:44.537178 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fa121-8e88-4324-bc80-8698e7aceda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:44.537451 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fa121-8e88-4324-bc80-8698e7aceda6 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:44.538172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fa121-8e88-4324-bc80-8698e7aceda6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:44.538496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fa121-8e88-4324-bc80-8698e7aceda6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:44.539879 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fa121-8e88-4324-bc80-8698e7aceda6 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:44.551096 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 266/511] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:44.551932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fa121-8e88-4324-bc80-8698e7aceda6 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:44.574550 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 246/512] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:44.575516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd67548-2b99-4392-aa33-4c3b4924322e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:44.650585 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fa121-8e88-4324-bc80-8698e7aceda6 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:44.671463 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd67548-2b99-4392-aa33-4c3b4924322e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:44.686010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba3334-27cf-4bae-9947-448a45c0faf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:44.686688 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba3334-27cf-4bae-9947-448a45c0faf7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:44.686956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba3334-27cf-4bae-9947-448a45c0faf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:44.687243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba3334-27cf-4bae-9947-448a45c0faf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:44.703860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3a68f-ab59-4326-9e5f-7f9cdf30f8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:44.704521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3a68f-ab59-4326-9e5f-7f9cdf30f8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:44.704799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3a68f-ab59-4326-9e5f-7f9cdf30f8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:44.705056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3a68f-ab59-4326-9e5f-7f9cdf30f8f3 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:44.705887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3a68f-ab59-4326-9e5f-7f9cdf30f8f3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:44.706191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3a68f-ab59-4326-9e5f-7f9cdf30f8f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:44.708029 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3a68f-ab59-4326-9e5f-7f9cdf30f8f3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:44.710891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:44.744726 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 247/513] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jul 5 09:59:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:44.745559 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3a68f-ab59-4326-9e5f-7f9cdf30f8f3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:44.749045 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:44.771867 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba3334-27cf-4bae-9947-448a45c0faf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:44.773353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba3334-27cf-4bae-9947-448a45c0faf7 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:44.849949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3a68f-ab59-4326-9e5f-7f9cdf30f8f3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:44.868531 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 267/514] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:44.869581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba3334-27cf-4bae-9947-448a45c0faf7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:44.883986 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00555bff-8a93-4800-a86c-3cb1f288f8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:44.884789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00555bff-8a93-4800-a86c-3cb1f288f8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:44.885069 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00555bff-8a93-4800-a86c-3cb1f288f8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:44.885340 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00555bff-8a93-4800-a86c-3cb1f288f8a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:44.896570 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00555bff-8a93-4800-a86c-3cb1f288f8a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:44.898038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00555bff-8a93-4800-a86c-3cb1f288f8a7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-00555bff-8a93-4800-a86c-3cb1f288f8a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:44.914975 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:44.915811 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:44.916647 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:44.916647 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:44.916647 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:44.916647 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:44.916647 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 248/515] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:44.916935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00555bff-8a93-4800-a86c-3cb1f288f8a7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:44.929002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:44.971181 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:44.971470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba3334-27cf-4bae-9947-448a45c0faf7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:45.006417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09813fb-1029-41f6-b811-19a7c9926037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:45.007085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09813fb-1029-41f6-b811-19a7c9926037 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:45.007369 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09813fb-1029-41f6-b811-19a7c9926037 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:45.007612 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09813fb-1029-41f6-b811-19a7c9926037 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:45.008314 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09813fb-1029-41f6-b811-19a7c9926037 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:45.009749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09813fb-1029-41f6-b811-19a7c9926037 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:45.013196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00555bff-8a93-4800-a86c-3cb1f288f8a7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:45.030885 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 268/516] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 09:59:44 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 09:59:45.031699 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09813fb-1029-41f6-b811-19a7c9926037 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:45.049212 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbc665-eb19-4a33-91b4-4c0c542f801d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:45.050060 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc665-eb19-4a33-91b4-4c0c542f801d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:45.050439 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc665-eb19-4a33-91b4-4c0c542f801d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:45.050714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc665-eb19-4a33-91b4-4c0c542f801d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:45.062535 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbc665-eb19-4a33-91b4-4c0c542f801d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:45.064319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbc665-eb19-4a33-91b4-4c0c542f801d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-86cbc665-eb19-4a33-91b4-4c0c542f801d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:45.090610 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:45.091712 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:45.092765 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:45.092765 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:45.092765 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:45.092765 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:45.095641 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 249/517] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:45.096689 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbc665-eb19-4a33-91b4-4c0c542f801d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:45.163915 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09813fb-1029-41f6-b811-19a7c9926037 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:45.175545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:45.202551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72aaacc-cf8e-4d58-8a46-b00834e44ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:45.203457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aaacc-cf8e-4d58-8a46-b00834e44ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:45.204147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aaacc-cf8e-4d58-8a46-b00834e44ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:45.204147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aaacc-cf8e-4d58-8a46-b00834e44ce1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:45.225620 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbc665-eb19-4a33-91b4-4c0c542f801d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:45.239090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:45.267865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12521c96-2349-457b-ace6-a3e571f38449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:45.268668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12521c96-2349-457b-ace6-a3e571f38449 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:45.268949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12521c96-2349-457b-ace6-a3e571f38449 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:45.269230 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12521c96-2349-457b-ace6-a3e571f38449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:45.279331 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72aaacc-cf8e-4d58-8a46-b00834e44ce1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:45.279467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12521c96-2349-457b-ace6-a3e571f38449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:45.280797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72aaacc-cf8e-4d58-8a46-b00834e44ce1 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:45.280899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12521c96-2349-457b-ace6-a3e571f38449 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-12521c96-2349-457b-ace6-a3e571f38449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:45.298619 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:45.299519 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:45.300383 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:45.300383 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:45.300383 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:45.300383 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:45.300383 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 250/518] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:45.303051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12521c96-2349-457b-ace6-a3e571f38449 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:45.384261 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 269/519] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:45.385136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72aaacc-cf8e-4d58-8a46-b00834e44ce1 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:45.403595 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:45.424149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12521c96-2349-457b-ace6-a3e571f38449 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:45.461335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b55df-79dc-46aa-beb9-2b68cd92da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:45.462012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b55df-79dc-46aa-beb9-2b68cd92da31 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:45.462326 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b55df-79dc-46aa-beb9-2b68cd92da31 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:45.462692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b55df-79dc-46aa-beb9-2b68cd92da31 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:45.463432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b55df-79dc-46aa-beb9-2b68cd92da31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:45.463748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b55df-79dc-46aa-beb9-2b68cd92da31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:45.465142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b55df-79dc-46aa-beb9-2b68cd92da31 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:45.481471 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 251/520] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:45.481471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b55df-79dc-46aa-beb9-2b68cd92da31 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:45.501743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:45.502582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72aaacc-cf8e-4d58-8a46-b00834e44ce1 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:45.541926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd70add-6542-4a27-aeb8-098b5300ab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:45.542623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd70add-6542-4a27-aeb8-098b5300ab8c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:45.542902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd70add-6542-4a27-aeb8-098b5300ab8c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:45.543172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd70add-6542-4a27-aeb8-098b5300ab8c None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:45.543887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd70add-6542-4a27-aeb8-098b5300ab8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:45.544187 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd70add-6542-4a27-aeb8-098b5300ab8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:45.545575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd70add-6542-4a27-aeb8-098b5300ab8c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:45.557227 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 270/521] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:45.562129 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd70add-6542-4a27-aeb8-098b5300ab8c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:45.584876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b55df-79dc-46aa-beb9-2b68cd92da31 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:45.622036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e63543-5f14-4bf5-815c-9f75eb63a441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:45.622837 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e63543-5f14-4bf5-815c-9f75eb63a441 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:45.623122 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e63543-5f14-4bf5-815c-9f75eb63a441 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:45.623401 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e63543-5f14-4bf5-815c-9f75eb63a441 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:45.624130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e63543-5f14-4bf5-815c-9f75eb63a441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:45.624438 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e63543-5f14-4bf5-815c-9f75eb63a441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:45.625931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e63543-5f14-4bf5-815c-9f75eb63a441 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:45.629557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:45.643603 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 252/522] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:45.643603 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e63543-5f14-4bf5-815c-9f75eb63a441 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:45.663626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd70add-6542-4a27-aeb8-098b5300ab8c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:45.699739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b948b7f-c416-4794-916f-0aefb0ff2525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:45.700882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b948b7f-c416-4794-916f-0aefb0ff2525 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:45.701174 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b948b7f-c416-4794-916f-0aefb0ff2525 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:45.701450 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b948b7f-c416-4794-916f-0aefb0ff2525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:45.712974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b948b7f-c416-4794-916f-0aefb0ff2525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:45.714469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b948b7f-c416-4794-916f-0aefb0ff2525 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:45.729043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-2b948b7f-c416-4794-916f-0aefb0ff2525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:45.731713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:45.732855 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:45.733730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:45.733730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:45.733730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:45.733730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:45.733730 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 271/523] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:45.734009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b948b7f-c416-4794-916f-0aefb0ff2525 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:45.754432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e63543-5f14-4bf5-815c-9f75eb63a441 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:45.800325 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce316da-b038-402d-8030-fbbc4c239d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:45.801223 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce316da-b038-402d-8030-fbbc4c239d12 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:45.801568 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce316da-b038-402d-8030-fbbc4c239d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:45.802056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce316da-b038-402d-8030-fbbc4c239d12 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:45.810123 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce316da-b038-402d-8030-fbbc4c239d12 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:45.811613 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce316da-b038-402d-8030-fbbc4c239d12 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9ce316da-b038-402d-8030-fbbc4c239d12 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:45.827520 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:45.828457 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:45.829486 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:45.829486 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:45.829486 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:45.829486 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:45.829486 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:45.829486 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:45.829486 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:45.829486 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:45.829486 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 09:59:45.829486 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:45.833621 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 253/524] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:45.834486 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce316da-b038-402d-8030-fbbc4c239d12 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:45.844082 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b948b7f-c416-4794-916f-0aefb0ff2525 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:45.867873 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:45.879449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1772d8-95b6-44cd-94ca-f78c77353497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:45.880310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1772d8-95b6-44cd-94ca-f78c77353497 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:45.880591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1772d8-95b6-44cd-94ca-f78c77353497 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:45.880885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1772d8-95b6-44cd-94ca-f78c77353497 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:45.887778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1772d8-95b6-44cd-94ca-f78c77353497 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:45.889216 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1772d8-95b6-44cd-94ca-f78c77353497 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-be1772d8-95b6-44cd-94ca-f78c77353497 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:45.903680 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:45.904543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:45.905382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:45.905382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:45.905382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:45.905382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:45.905382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:45.905382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:45.905382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:45.905382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:45.905382 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 272/525] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:45.905942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1772d8-95b6-44cd-94ca-f78c77353497 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:45.942822 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce316da-b038-402d-8030-fbbc4c239d12 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:45.977979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08a2c5-dd6b-47a5-b968-21d47aa04a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:45.978750 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a2c5-dd6b-47a5-b968-21d47aa04a73 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:45.979033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a2c5-dd6b-47a5-b968-21d47aa04a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:45.979361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a2c5-dd6b-47a5-b968-21d47aa04a73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:45.987042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08a2c5-dd6b-47a5-b968-21d47aa04a73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:45.988615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08a2c5-dd6b-47a5-b968-21d47aa04a73 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:45.991329 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5c08a2c5-dd6b-47a5-b968-21d47aa04a73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:46.004890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:46.005900 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:46.006823 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:46.006823 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:46.006823 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:46.006823 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:46.006823 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:46.006823 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:46.006823 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:46.006823 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:46.006823 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 254/526] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:46.007291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08a2c5-dd6b-47a5-b968-21d47aa04a73 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:46.009341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1772d8-95b6-44cd-94ca-f78c77353497 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:46.045689 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471111d6-d8be-47b8-bfc7-3ee91e368a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:46.046390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471111d6-d8be-47b8-bfc7-3ee91e368a48 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:46.046675 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471111d6-d8be-47b8-bfc7-3ee91e368a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:46.046949 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471111d6-d8be-47b8-bfc7-3ee91e368a48 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:46.047665 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471111d6-d8be-47b8-bfc7-3ee91e368a48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:46.047964 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471111d6-d8be-47b8-bfc7-3ee91e368a48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:46.049331 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471111d6-d8be-47b8-bfc7-3ee91e368a48 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:46.060404 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 273/527] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:46.061178 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471111d6-d8be-47b8-bfc7-3ee91e368a48 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:46.105869 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:46.126852 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08a2c5-dd6b-47a5-b968-21d47aa04a73 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:46.165091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ee9b6-cf3d-4fbd-85c7-11d499ff8d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:46.165833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ee9b6-cf3d-4fbd-85c7-11d499ff8d3f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:46.166067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ee9b6-cf3d-4fbd-85c7-11d499ff8d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:46.166374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ee9b6-cf3d-4fbd-85c7-11d499ff8d3f None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:46.167246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ee9b6-cf3d-4fbd-85c7-11d499ff8d3f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:46.167575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ee9b6-cf3d-4fbd-85c7-11d499ff8d3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:46.169052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ee9b6-cf3d-4fbd-85c7-11d499ff8d3f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:46.172707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471111d6-d8be-47b8-bfc7-3ee91e368a48 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:46.193947 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 255/528] 10.222.0.16 () {56 vars in 1139 bytes} [Tue Jul 5 09:59:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:46.193947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ee9b6-cf3d-4fbd-85c7-11d499ff8d3f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:46.216154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791eae6-f5de-493c-a0d9-4d474f297414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:46.216832 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791eae6-f5de-493c-a0d9-4d474f297414 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:46.217118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791eae6-f5de-493c-a0d9-4d474f297414 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:46.217397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791eae6-f5de-493c-a0d9-4d474f297414 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:46.218189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791eae6-f5de-493c-a0d9-4d474f297414 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:46.218522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791eae6-f5de-493c-a0d9-4d474f297414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:46.219998 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791eae6-f5de-493c-a0d9-4d474f297414 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:46.244202 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 274/529] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:46.244202 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791eae6-f5de-493c-a0d9-4d474f297414 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:46.260613 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:46.302492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ee9b6-cf3d-4fbd-85c7-11d499ff8d3f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:46.336864 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d2505-e63d-4623-9616-124788199f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:46.337528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d2505-e63d-4623-9616-124788199f08 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:46.337812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d2505-e63d-4623-9616-124788199f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:46.338090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d2505-e63d-4623-9616-124788199f08 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:46.338906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d2505-e63d-4623-9616-124788199f08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:46.339214 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d2505-e63d-4623-9616-124788199f08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:46.340681 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d2505-e63d-4623-9616-124788199f08 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:46.343011 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791eae6-f5de-493c-a0d9-4d474f297414 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:46.344538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:46.364101 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 256/530] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:46.364101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d2505-e63d-4623-9616-124788199f08 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:46.386776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a6974-009e-43a7-8f30-ee794fd26408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:46.388150 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a6974-009e-43a7-8f30-ee794fd26408 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:46.388453 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a6974-009e-43a7-8f30-ee794fd26408 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:46.388732 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a6974-009e-43a7-8f30-ee794fd26408 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:46.396551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a6974-009e-43a7-8f30-ee794fd26408 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:46.398908 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a6974-009e-43a7-8f30-ee794fd26408 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-a57a6974-009e-43a7-8f30-ee794fd26408 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:46.414690 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:46.415543 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:46.416430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:46.416430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:46.416430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:46.416430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:46.416430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:46.416430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:46.416430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:46.416430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:46.416430 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 275/531] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:46.416922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a6974-009e-43a7-8f30-ee794fd26408 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:46.477838 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d2505-e63d-4623-9616-124788199f08 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:46.496707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:46.514687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed28c8-bd18-4869-bf64-cf08ddae7103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:46.515358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed28c8-bd18-4869-bf64-cf08ddae7103 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:46.515642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed28c8-bd18-4869-bf64-cf08ddae7103 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:46.515909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed28c8-bd18-4869-bf64-cf08ddae7103 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:46.522196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a6974-009e-43a7-8f30-ee794fd26408 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:46.556356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cdca3e-367f-415f-b1ca-bab94fb5f8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:46.557020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdca3e-367f-415f-b1ca-bab94fb5f8ad None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:46.557306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdca3e-367f-415f-b1ca-bab94fb5f8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:46.557587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdca3e-367f-415f-b1ca-bab94fb5f8ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:46.558448 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cdca3e-367f-415f-b1ca-bab94fb5f8ad None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=fee6ee8f99c34aa281bf02ecd3f23216) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:46.558764 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cdca3e-367f-415f-b1ca-bab94fb5f8ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:46.560217 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cdca3e-367f-415f-b1ca-bab94fb5f8ad None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:46.566344 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 276/532] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:46 2022] GET /identity/v3/role_assignments?user.id=fee6ee8f99c34aa281bf02ecd3f23216&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:46.567443 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cdca3e-367f-415f-b1ca-bab94fb5f8ad None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:46.589459 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed28c8-bd18-4869-bf64-cf08ddae7103 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:46.591342 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed28c8-bd18-4869-bf64-cf08ddae7103 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:46.609977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:46.697530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cdca3e-367f-415f-b1ca-bab94fb5f8ad None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:46.737158 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 257/533] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:46.737721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532b64f-48a8-4f67-a706-4f25ee1f4c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:46.739842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532b64f-48a8-4f67-a706-4f25ee1f4c13 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:46.740310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532b64f-48a8-4f67-a706-4f25ee1f4c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:46.740757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532b64f-48a8-4f67-a706-4f25ee1f4c13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:46.742081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532b64f-48a8-4f67-a706-4f25ee1f4c13 None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=b97736e49e2a4bfb9cf7093cfa8b2b16) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:46.742387 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed28c8-bd18-4869-bf64-cf08ddae7103 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:46.742894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532b64f-48a8-4f67-a706-4f25ee1f4c13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:46.744527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532b64f-48a8-4f67-a706-4f25ee1f4c13 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:46.750828 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 277/534] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:46 2022] GET /identity/v3/role_assignments?user.id=b97736e49e2a4bfb9cf7093cfa8b2b16&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:46.751812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532b64f-48a8-4f67-a706-4f25ee1f4c13 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:46.785845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:46.880751 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:46.893235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532b64f-48a8-4f67-a706-4f25ee1f4c13 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:46.939443 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fa810-5249-46a1-ad24-40e1df8e7b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:46.940285 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fa810-5249-46a1-ad24-40e1df8e7b2f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:46.940566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fa810-5249-46a1-ad24-40e1df8e7b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:46.940837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fa810-5249-46a1-ad24-40e1df8e7b2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:46.953266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed28c8-bd18-4869-bf64-cf08ddae7103 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:46.972425 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fa810-5249-46a1-ad24-40e1df8e7b2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:46.973981 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fa810-5249-46a1-ad24-40e1df8e7b2f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:46.997218 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283bb1b-a451-4ebe-90b1-48a8cad99e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:46.997897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283bb1b-a451-4ebe-90b1-48a8cad99e9c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:46.998176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283bb1b-a451-4ebe-90b1-48a8cad99e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:46.998476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283bb1b-a451-4ebe-90b1-48a8cad99e9c None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:46.999305 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283bb1b-a451-4ebe-90b1-48a8cad99e9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:46.999607 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283bb1b-a451-4ebe-90b1-48a8cad99e9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:47.001096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283bb1b-a451-4ebe-90b1-48a8cad99e9c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-f48fa810-5249-46a1-ad24-40e1df8e7b2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:47.001959 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:47.002789 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:47.004042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:47.004042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:47.004042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:47.004042 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:47.004042 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 278/535] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:47.009056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fa810-5249-46a1-ad24-40e1df8e7b2f None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:47.015243 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 258/536] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:47.021443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283bb1b-a451-4ebe-90b1-48a8cad99e9c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:47.134595 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:47.154466 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283bb1b-a451-4ebe-90b1-48a8cad99e9c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:47.155140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fa810-5249-46a1-ad24-40e1df8e7b2f None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:47.191048 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b5ab4-2fb3-498e-bfdd-f863847f1382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:47.192230 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5ab4-2fb3-498e-bfdd-f863847f1382 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:47.192549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5ab4-2fb3-498e-bfdd-f863847f1382 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:47.192823 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5ab4-2fb3-498e-bfdd-f863847f1382 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:47.193660 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b5ab4-2fb3-498e-bfdd-f863847f1382 None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=dc11037d649241608ea1378cf4692dad) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:47.193969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b5ab4-2fb3-498e-bfdd-f863847f1382 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:47.194887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c46f15-21d2-443d-931e-b29e73a5b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:47.195517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b5ab4-2fb3-498e-bfdd-f863847f1382 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:47.195927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c46f15-21d2-443d-931e-b29e73a5b928 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:47.196386 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c46f15-21d2-443d-931e-b29e73a5b928 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:47.196819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c46f15-21d2-443d-931e-b29e73a5b928 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:47.197809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c46f15-21d2-443d-931e-b29e73a5b928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:47.198401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c46f15-21d2-443d-931e-b29e73a5b928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:47.199973 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c46f15-21d2-443d-931e-b29e73a5b928 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:47.206243 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 259/537] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:47 2022] GET /identity/v3/role_assignments?user.id=dc11037d649241608ea1378cf4692dad&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:47.206243 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b037e9-a4a2-4147-a25d-a57c7e7be95c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:47.206243 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b037e9-a4a2-4147-a25d-a57c7e7be95c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:47.206243 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b037e9-a4a2-4147-a25d-a57c7e7be95c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:47.209347 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 260/538] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:47.212571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1185c-8c68-4650-bc9f-82a456dc1f36 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:47.212571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1185c-8c68-4650-bc9f-82a456dc1f36 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:47.212571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1185c-8c68-4650-bc9f-82a456dc1f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:47.228993 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 279/539] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:47.232563 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b3c1185c-8c68-4650-bc9f-82a456dc1f36 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:47.232563 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b3c1185c-8c68-4650-bc9f-82a456dc1f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:47.234646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c46f15-21d2-443d-931e-b29e73a5b928 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:47.240224 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:47.305703 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 261/540] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:47.328882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1185c-8c68-4650-bc9f-82a456dc1f36 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:47.410830 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c46f15-21d2-443d-931e-b29e73a5b928 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:47.431806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:47.448354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2d3bc-ceab-4496-b1cc-27156c5a6c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:47.449171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2d3bc-ceab-4496-b1cc-27156c5a6c45 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:47.449456 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2d3bc-ceab-4496-b1cc-27156c5a6c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:47.449734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2d3bc-ceab-4496-b1cc-27156c5a6c45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:47.456792 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2d3bc-ceab-4496-b1cc-27156c5a6c45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:47.458686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2d3bc-ceab-4496-b1cc-27156c5a6c45 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-e9e2d3bc-ceab-4496-b1cc-27156c5a6c45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:47.480378 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:47.481272 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:47.482098 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:47.482098 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:47.482098 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:47.482098 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:47.482098 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:47.482098 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:47.482098 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:47.482098 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:47.482098 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 280/541] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:47.487533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2d3bc-ceab-4496-b1cc-27156c5a6c45 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:47.511438 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1185c-8c68-4650-bc9f-82a456dc1f36 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:47.590692 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:47.608059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2d3bc-ceab-4496-b1cc-27156c5a6c45 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:47.631503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfe678-23ab-4ee3-85b3-0c022ae3b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:47.632329 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfe678-23ab-4ee3-85b3-0c022ae3b35a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:47.632614 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfe678-23ab-4ee3-85b3-0c022ae3b35a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:47.632888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfe678-23ab-4ee3-85b3-0c022ae3b35a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:47.642167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c0d25-b987-4d04-9f90-974a562e8af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:47.642865 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c0d25-b987-4d04-9f90-974a562e8af3 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:47.643261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c0d25-b987-4d04-9f90-974a562e8af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:47.643543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c0d25-b987-4d04-9f90-974a562e8af3 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:47.644342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c0d25-b987-4d04-9f90-974a562e8af3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:47.644708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c0d25-b987-4d04-9f90-974a562e8af3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:47.646270 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c0d25-b987-4d04-9f90-974a562e8af3 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:47.658871 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 281/542] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:47.664532 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c0d25-b987-4d04-9f90-974a562e8af3 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:47.673254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfe678-23ab-4ee3-85b3-0c022ae3b35a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d3cfe678-23ab-4ee3-85b3-0c022ae3b35a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:47.681188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:47.682078 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:47.682894 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:47.690886 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 262/543] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:47.699172 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfe678-23ab-4ee3-85b3-0c022ae3b35a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:47.775683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c0d25-b987-4d04-9f90-974a562e8af3 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:47.802599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:47.812966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80bfaf-4d53-4eb4-9941-b0ca5676f29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:47.813706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80bfaf-4d53-4eb4-9941-b0ca5676f29c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:47.814055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80bfaf-4d53-4eb4-9941-b0ca5676f29c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:47.814530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80bfaf-4d53-4eb4-9941-b0ca5676f29c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:47.815436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80bfaf-4d53-4eb4-9941-b0ca5676f29c None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=e0865f21da0a4c95a84c0099a1b9a78b) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:47.815813 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80bfaf-4d53-4eb4-9941-b0ca5676f29c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:47.817358 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80bfaf-4d53-4eb4-9941-b0ca5676f29c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:47.824169 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 282/544] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:47 2022] GET /identity/v3/role_assignments?user.id=e0865f21da0a4c95a84c0099a1b9a78b&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:47.941176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfe678-23ab-4ee3-85b3-0c022ae3b35a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:47.984355 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:47.990339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325afe3-d478-421f-8518-72f4206c304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:47.991107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325afe3-d478-421f-8518-72f4206c304f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:47.991399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325afe3-d478-421f-8518-72f4206c304f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:47.991675 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325afe3-d478-421f-8518-72f4206c304f None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:47.992434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325afe3-d478-421f-8518-72f4206c304f None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:47.992738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325afe3-d478-421f-8518-72f4206c304f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:47.994133 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325afe3-d478-421f-8518-72f4206c304f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:48.003453 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 263/545] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:48.008779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80bfaf-4d53-4eb4-9941-b0ca5676f29c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:48.127479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80bfaf-4d53-4eb4-9941-b0ca5676f29c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:48.160823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c27e7c-7d01-491f-8b08-220ac976990a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:48.161928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27e7c-7d01-491f-8b08-220ac976990a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:48.162467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27e7c-7d01-491f-8b08-220ac976990a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:48.162936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27e7c-7d01-491f-8b08-220ac976990a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:48.173719 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c27e7c-7d01-491f-8b08-220ac976990a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:48.175446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c27e7c-7d01-491f-8b08-220ac976990a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-a2c27e7c-7d01-491f-8b08-220ac976990a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:48.193065 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:48.193943 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:48.194857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:48.194857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:48.194857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:48.194857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:48.196730 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 283/546] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:48.207301 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325afe3-d478-421f-8518-72f4206c304f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:48.337302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325afe3-d478-421f-8518-72f4206c304f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:48.385738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd54a0-b32f-47b9-81e4-cd2b0143bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:48.386446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd54a0-b32f-47b9-81e4-cd2b0143bd9b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:48.386738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd54a0-b32f-47b9-81e4-cd2b0143bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:48.387021 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd54a0-b32f-47b9-81e4-cd2b0143bd9b None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:48.387758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd54a0-b32f-47b9-81e4-cd2b0143bd9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:48.388070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd54a0-b32f-47b9-81e4-cd2b0143bd9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:48.389538 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd54a0-b32f-47b9-81e4-cd2b0143bd9b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:48.406840 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 264/547] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:48.412167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c27e7c-7d01-491f-8b08-220ac976990a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:48.473859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:48.543461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c27e7c-7d01-491f-8b08-220ac976990a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:48.597384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705960c-7e33-47d7-8916-96de82b9196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:48.598244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705960c-7e33-47d7-8916-96de82b9196a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:48.598558 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705960c-7e33-47d7-8916-96de82b9196a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:48.598842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705960c-7e33-47d7-8916-96de82b9196a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:48.606713 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705960c-7e33-47d7-8916-96de82b9196a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:48.608305 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705960c-7e33-47d7-8916-96de82b9196a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-1705960c-7e33-47d7-8916-96de82b9196a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:48.629216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:48.630205 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:48.631745 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:48.631745 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:48.631745 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:48.631745 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:48.631745 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:48.631745 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:48.631745 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:48.631745 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:48.631745 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 09:59:48.631745 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:48.631745 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 284/548] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:59:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:48.636618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd54a0-b32f-47b9-81e4-cd2b0143bd9b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:48.726287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:48.793859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd54a0-b32f-47b9-81e4-cd2b0143bd9b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:48.842692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449994c4-09b0-42d8-810f-f5db4489f639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:48.843370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449994c4-09b0-42d8-810f-f5db4489f639 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:48.843652 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449994c4-09b0-42d8-810f-f5db4489f639 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:48.843925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449994c4-09b0-42d8-810f-f5db4489f639 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:48.844764 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449994c4-09b0-42d8-810f-f5db4489f639 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:48.845077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449994c4-09b0-42d8-810f-f5db4489f639 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:48.850433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449994c4-09b0-42d8-810f-f5db4489f639 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:48.872783 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 265/549] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jul 5 09:59:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:48.878326 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705960c-7e33-47d7-8916-96de82b9196a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:49.017152 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705960c-7e33-47d7-8916-96de82b9196a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:49.021053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:49.049044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7930f-4048-43d2-9f9c-b2b5cc88c38d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.049370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7930f-4048-43d2-9f9c-b2b5cc88c38d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.050615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7930f-4048-43d2-9f9c-b2b5cc88c38d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.052174 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 266/550] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:49.063673 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d540a2-68fe-49ec-b8c6-f0d1e0009abb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.063975 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d540a2-68fe-49ec-b8c6-f0d1e0009abb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.064232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d540a2-68fe-49ec-b8c6-f0d1e0009abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.067708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd52a2-be09-4841-9404-74d6da932b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:49.068650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd52a2-be09-4841-9404-74d6da932b3c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.069135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd52a2-be09-4841-9404-74d6da932b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.069648 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd52a2-be09-4841-9404-74d6da932b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.100920 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-52d540a2-68fe-49ec-b8c6-f0d1e0009abb None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:49.102160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-52d540a2-68fe-49ec-b8c6-f0d1e0009abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:49.164409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd52a2-be09-4841-9404-74d6da932b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:49.165866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd52a2-be09-4841-9404-74d6da932b3c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:49.254998 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 267/551] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:49.257965 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0497e0a-0e92-4221-bc07-ddf48085b5e6 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.257965 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0497e0a-0e92-4221-bc07-ddf48085b5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.257965 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0497e0a-0e92-4221-bc07-ddf48085b5e6 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.257965 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 268/552] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:49.265452 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba0df6-555e-4032-917e-ea32bf0d3c80 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.265735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba0df6-555e-4032-917e-ea32bf0d3c80 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.266017 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba0df6-555e-4032-917e-ea32bf0d3c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.277465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:49.285501 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 285/553] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:49.286874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd52a2-be09-4841-9404-74d6da932b3c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:49.299698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e5ba0df6-555e-4032-917e-ea32bf0d3c80 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:49.301541 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e5ba0df6-555e-4032-917e-ea32bf0d3c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:49.348540 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:49.394389 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 269/554] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:49.395528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba0df6-555e-4032-917e-ea32bf0d3c80 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:49.403542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd52a2-be09-4841-9404-74d6da932b3c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:49.444815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871483c-5bdb-4e0c-b9e5-2fee45149aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:49.445773 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871483c-5bdb-4e0c-b9e5-2fee45149aea None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.446149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871483c-5bdb-4e0c-b9e5-2fee45149aea None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.446547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871483c-5bdb-4e0c-b9e5-2fee45149aea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.457538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871483c-5bdb-4e0c-b9e5-2fee45149aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-4871483c-5bdb-4e0c-b9e5-2fee45149aea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:49.466057 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:49.467376 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:49.468414 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:49.470373 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 286/555] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:49.475361 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871483c-5bdb-4e0c-b9e5-2fee45149aea None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:49.513553 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba0df6-555e-4032-917e-ea32bf0d3c80 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:49.531778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:49.553252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea1f05-f67c-4dc0-91c2-c56a33d72919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:49.554302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea1f05-f67c-4dc0-91c2-c56a33d72919 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.554733 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea1f05-f67c-4dc0-91c2-c56a33d72919 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.555185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea1f05-f67c-4dc0-91c2-c56a33d72919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.566392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea1f05-f67c-4dc0-91c2-c56a33d72919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:49.568126 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea1f05-f67c-4dc0-91c2-c56a33d72919 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:49.575405 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871483c-5bdb-4e0c-b9e5-2fee45149aea None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5fea1f05-f67c-4dc0-91c2-c56a33d72919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:49.588177 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:49.589146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:49.590042 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:49.590042 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:49.590042 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:49.590042 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:49.592412 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 270/556] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:49.593385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea1f05-f67c-4dc0-91c2-c56a33d72919 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:49.615185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436aa72-e859-4979-96db-31ab5bfbd5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:49.615993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436aa72-e859-4979-96db-31ab5bfbd5fb None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.616271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436aa72-e859-4979-96db-31ab5bfbd5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.616536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436aa72-e859-4979-96db-31ab5bfbd5fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.620563 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:49.627233 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436aa72-e859-4979-96db-31ab5bfbd5fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-7436aa72-e859-4979-96db-31ab5bfbd5fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:49.635578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:49.636495 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 287/557] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:49.638694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436aa72-e859-4979-96db-31ab5bfbd5fb None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:49.704014 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea1f05-f67c-4dc0-91c2-c56a33d72919 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:49.740340 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436aa72-e859-4979-96db-31ab5bfbd5fb None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:49.742507 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50fea0-89bb-4d16-8487-dafa33c4afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:49.743188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50fea0-89bb-4d16-8487-dafa33c4afd1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.743470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50fea0-89bb-4d16-8487-dafa33c4afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.743744 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50fea0-89bb-4d16-8487-dafa33c4afd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.744491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca50fea0-89bb-4d16-8487-dafa33c4afd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:49.744802 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca50fea0-89bb-4d16-8487-dafa33c4afd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:49.746170 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca50fea0-89bb-4d16-8487-dafa33c4afd1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:49.757714 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 271/558] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:49.757714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fc927-a0f1-4dd7-a1ce-3603076dc08e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.757714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fc927-a0f1-4dd7-a1ce-3603076dc08e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.757714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fc927-a0f1-4dd7-a1ce-3603076dc08e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.757714 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 272/559] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:49.758474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fc927-a0f1-4dd7-a1ce-3603076dc08e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:49.767276 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:49.786831 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c98a7-2e75-4f0f-8dd8-de372d964040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:49.787437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c98a7-2e75-4f0f-8dd8-de372d964040 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.787715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c98a7-2e75-4f0f-8dd8-de372d964040 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.787992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c98a7-2e75-4f0f-8dd8-de372d964040 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.788715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c98a7-2e75-4f0f-8dd8-de372d964040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:49.789019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c98a7-2e75-4f0f-8dd8-de372d964040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:49.790429 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c98a7-2e75-4f0f-8dd8-de372d964040 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:49.803019 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 288/560] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:49.804363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0a42-58a6-4f24-9983-bc0f9ae33b2d None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.804654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0a42-58a6-4f24-9983-bc0f9ae33b2d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.804926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0a42-58a6-4f24-9983-bc0f9ae33b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.822512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-588a0a42-58a6-4f24-9983-bc0f9ae33b2d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:49.823704 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-588a0a42-58a6-4f24-9983-bc0f9ae33b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:49.855039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:49.872552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fc927-a0f1-4dd7-a1ce-3603076dc08e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:49.900178 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 289/561] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:49.900178 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a0a42-58a6-4f24-9983-bc0f9ae33b2d None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:49.907858 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6d45e-1ee6-4aad-b286-df554916d3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:49.908619 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6d45e-1ee6-4aad-b286-df554916d3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:49.908979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6d45e-1ee6-4aad-b286-df554916d3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:49.909325 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6d45e-1ee6-4aad-b286-df554916d3d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:49.910092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6d45e-1ee6-4aad-b286-df554916d3d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:49.910437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6d45e-1ee6-4aad-b286-df554916d3d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:49.911898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6d45e-1ee6-4aad-b286-df554916d3d9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:49.920938 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 273/562] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:49.921764 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6d45e-1ee6-4aad-b286-df554916d3d9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.005599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a0a42-58a6-4f24-9983-bc0f9ae33b2d None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.014790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:50.025470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6d45e-1ee6-4aad-b286-df554916d3d9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.042454 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6be61-b963-4c27-a9f1-08e5f45594cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.043204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6be61-b963-4c27-a9f1-08e5f45594cc None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.043485 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6be61-b963-4c27-a9f1-08e5f45594cc None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.043763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6be61-b963-4c27-a9f1-08e5f45594cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.054557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6be61-b963-4c27-a9f1-08e5f45594cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:50.056006 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6be61-b963-4c27-a9f1-08e5f45594cc None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:50.065658 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3b652-f4e1-40ad-85ae-b3d5cfdca2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.066699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3b652-f4e1-40ad-85ae-b3d5cfdca2de None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.067392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3b652-f4e1-40ad-85ae-b3d5cfdca2de None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.067823 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3b652-f4e1-40ad-85ae-b3d5cfdca2de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.075015 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3b652-f4e1-40ad-85ae-b3d5cfdca2de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:50.076739 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3b652-f4e1-40ad-85ae-b3d5cfdca2de None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-86a6be61-b963-4c27-a9f1-08e5f45594cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:50.077619 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:50.078659 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:50.079667 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:50.079667 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:50.079667 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:50.079667 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:50.079667 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 290/563] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:50.080080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6be61-b963-4c27-a9f1-08e5f45594cc None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-8af3b652-f4e1-40ad-85ae-b3d5cfdca2de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:50.101022 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:50.102009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:50.102909 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:50.102909 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:50.102909 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:50.102909 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:50.102909 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:50.102909 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:50.102909 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:50.102909 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:50.102909 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 274/564] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:50.104173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3b652-f4e1-40ad-85ae-b3d5cfdca2de None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.117062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:50.188869 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6be61-b963-4c27-a9f1-08e5f45594cc None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.210008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3b652-f4e1-40ad-85ae-b3d5cfdca2de None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.225635 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d76d93-e486-4f27-9401-b5474b043106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.226517 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d76d93-e486-4f27-9401-b5474b043106 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.226847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d76d93-e486-4f27-9401-b5474b043106 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.227140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d76d93-e486-4f27-9401-b5474b043106 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.237333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d76d93-e486-4f27-9401-b5474b043106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:50.244078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848479b-400b-4418-a3a1-dfd43542ebab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.244878 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848479b-400b-4418-a3a1-dfd43542ebab None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-17d76d93-e486-4f27-9401-b5474b043106 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:50.245114 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:50.246259 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:50.247130 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:50.247758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848479b-400b-4418-a3a1-dfd43542ebab None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.247758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848479b-400b-4418-a3a1-dfd43542ebab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.247891 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 291/565] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:50.247891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d76d93-e486-4f27-9401-b5474b043106 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.259699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848479b-400b-4418-a3a1-dfd43542ebab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:50.261494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848479b-400b-4418-a3a1-dfd43542ebab None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:50.268948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a848479b-400b-4418-a3a1-dfd43542ebab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:50.280525 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:50.281585 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:50.282549 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:50.282549 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:50.282549 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:50.282549 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:50.282549 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 275/566] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:50.282549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848479b-400b-4418-a3a1-dfd43542ebab None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.341254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:50.348002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d76d93-e486-4f27-9401-b5474b043106 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.379325 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848479b-400b-4418-a3a1-dfd43542ebab None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.382281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819d714-e761-4bff-904b-0b24ad0ea22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.382977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819d714-e761-4bff-904b-0b24ad0ea22e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.383259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819d714-e761-4bff-904b-0b24ad0ea22e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.383532 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819d714-e761-4bff-904b-0b24ad0ea22e None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.384254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819d714-e761-4bff-904b-0b24ad0ea22e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:50.384556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819d714-e761-4bff-904b-0b24ad0ea22e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:50.385963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819d714-e761-4bff-904b-0b24ad0ea22e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:50.398668 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 292/567] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:50.399896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d58c22-dabb-4e95-a3d9-4c6b451b2692 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.400239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d58c22-dabb-4e95-a3d9-4c6b451b2692 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.400499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d58c22-dabb-4e95-a3d9-4c6b451b2692 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.401866 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 293/568] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:50.406355 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d58c22-dabb-4e95-a3d9-4c6b451b2692 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.413725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5652666-2367-499a-9beb-fd61462ed6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.414585 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5652666-2367-499a-9beb-fd61462ed6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.415061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5652666-2367-499a-9beb-fd61462ed6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.415318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5652666-2367-499a-9beb-fd61462ed6b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.416191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5652666-2367-499a-9beb-fd61462ed6b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:50.416537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5652666-2367-499a-9beb-fd61462ed6b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:50.418682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5652666-2367-499a-9beb-fd61462ed6b7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:50.430573 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 276/569] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:50.431392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5652666-2367-499a-9beb-fd61462ed6b7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.487775 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:50.508781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d58c22-dabb-4e95-a3d9-4c6b451b2692 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.529111 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5652666-2367-499a-9beb-fd61462ed6b7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.542619 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965075a8-025c-43ca-847b-14874ac880a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.543291 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965075a8-025c-43ca-847b-14874ac880a4 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.543572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965075a8-025c-43ca-847b-14874ac880a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.543846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965075a8-025c-43ca-847b-14874ac880a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.544591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965075a8-025c-43ca-847b-14874ac880a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:50.544899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965075a8-025c-43ca-847b-14874ac880a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:50.546303 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965075a8-025c-43ca-847b-14874ac880a4 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:50.547687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:50.554830 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 294/570] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:50.555619 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965075a8-025c-43ca-847b-14874ac880a4 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.562644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9e71d-fdfd-496d-9e9c-dbadc5223ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.563308 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9e71d-fdfd-496d-9e9c-dbadc5223ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.563589 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9e71d-fdfd-496d-9e9c-dbadc5223ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.563863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9e71d-fdfd-496d-9e9c-dbadc5223ac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.564984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9e71d-fdfd-496d-9e9c-dbadc5223ac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:50.565291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9e71d-fdfd-496d-9e9c-dbadc5223ac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:50.566685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9e71d-fdfd-496d-9e9c-dbadc5223ac9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:50.578264 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 277/571] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:50.579431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff7167-08a7-4316-bdb3-1f6421c04a38 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.579714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff7167-08a7-4316-bdb3-1f6421c04a38 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.579987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff7167-08a7-4316-bdb3-1f6421c04a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.596101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e7ff7167-08a7-4316-bdb3-1f6421c04a38 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:50.597221 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e7ff7167-08a7-4316-bdb3-1f6421c04a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:50.657680 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965075a8-025c-43ca-847b-14874ac880a4 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.660414 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 278/572] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:50.671222 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff7167-08a7-4316-bdb3-1f6421c04a38 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.692662 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fe859-05b0-4405-a406-fc3c6d276d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.693462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe859-05b0-4405-a406-fc3c6d276d85 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.694069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe859-05b0-4405-a406-fc3c6d276d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.694386 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe859-05b0-4405-a406-fc3c6d276d85 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.701818 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fe859-05b0-4405-a406-fc3c6d276d85 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:50.703290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fe859-05b0-4405-a406-fc3c6d276d85 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-dc0fe859-05b0-4405-a406-fc3c6d276d85 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:50.717300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:50.718321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 295/573] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jul 5 09:59:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:50.719374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fe859-05b0-4405-a406-fc3c6d276d85 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.722327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:50.779693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff7167-08a7-4316-bdb3-1f6421c04a38 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.790130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:50.813693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd55cf-96e7-4823-8f9b-19b0114247b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.814375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd55cf-96e7-4823-8f9b-19b0114247b1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.814653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd55cf-96e7-4823-8f9b-19b0114247b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.814927 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd55cf-96e7-4823-8f9b-19b0114247b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.815732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd55cf-96e7-4823-8f9b-19b0114247b1 None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=dbfded29b70140a387f7b8c07c296861) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:50.816047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd55cf-96e7-4823-8f9b-19b0114247b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:50.817473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd55cf-96e7-4823-8f9b-19b0114247b1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:50.822790 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 279/574] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:50 2022] GET /identity/v3/role_assignments?user.id=dbfded29b70140a387f7b8c07c296861&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:50.823548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd55cf-96e7-4823-8f9b-19b0114247b1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.825100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fe859-05b0-4405-a406-fc3c6d276d85 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.864278 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329416da-374b-425f-8372-0033430f51d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.865017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329416da-374b-425f-8372-0033430f51d3 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.865298 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329416da-374b-425f-8372-0033430f51d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.865573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329416da-374b-425f-8372-0033430f51d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:50.878987 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329416da-374b-425f-8372-0033430f51d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:50.880520 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329416da-374b-425f-8372-0033430f51d3 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-329416da-374b-425f-8372-0033430f51d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:50.898863 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:50.899879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:50.900772 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:50.900772 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:50.900772 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:50.900772 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:50.900772 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 296/575] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:50.901055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329416da-374b-425f-8372-0033430f51d3 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:50.947360 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd55cf-96e7-4823-8f9b-19b0114247b1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:50.963645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:50.991156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6caea7-28d3-4ada-ad7e-9efc3d1256d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:50.992356 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6caea7-28d3-4ada-ad7e-9efc3d1256d0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:50.992706 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6caea7-28d3-4ada-ad7e-9efc3d1256d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:50.993304 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6caea7-28d3-4ada-ad7e-9efc3d1256d0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.002653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6caea7-28d3-4ada-ad7e-9efc3d1256d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:51.004429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6caea7-28d3-4ada-ad7e-9efc3d1256d0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:51.016660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329416da-374b-425f-8372-0033430f51d3 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-0d6caea7-28d3-4ada-ad7e-9efc3d1256d0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:51.019928 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:51.021171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:51.022363 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:51.022363 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:51.022363 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:51.022363 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:51.022363 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:51.022363 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:51.022363 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:51.022363 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:51.022363 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 05 09:59:51.022363 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:51.027743 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 280/576] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:51.027743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6caea7-28d3-4ada-ad7e-9efc3d1256d0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:51.057105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f1704-7010-441a-ab67-10ca62ad8103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:51.058014 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1704-7010-441a-ab67-10ca62ad8103 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:51.058322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1704-7010-441a-ab67-10ca62ad8103 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:51.058678 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1704-7010-441a-ab67-10ca62ad8103 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.061863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:51.070473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f1704-7010-441a-ab67-10ca62ad8103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-ff2f1704-7010-441a-ab67-10ca62ad8103 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:51.078599 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:51.079518 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 297/577] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:51.086831 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f1704-7010-441a-ab67-10ca62ad8103 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:51.142368 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6caea7-28d3-4ada-ad7e-9efc3d1256d0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:51.178922 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee2a35-b156-4efe-a14b-dab1b93ff137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:51.179606 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee2a35-b156-4efe-a14b-dab1b93ff137 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:51.179891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee2a35-b156-4efe-a14b-dab1b93ff137 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:51.180170 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee2a35-b156-4efe-a14b-dab1b93ff137 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.181013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee2a35-b156-4efe-a14b-dab1b93ff137 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:51.181327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee2a35-b156-4efe-a14b-dab1b93ff137 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:51.183121 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee2a35-b156-4efe-a14b-dab1b93ff137 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:51.199991 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f1704-7010-441a-ab67-10ca62ad8103 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:51.205328 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 281/578] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jul 5 09:59:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:51.205328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee2a35-b156-4efe-a14b-dab1b93ff137 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:51.220553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:51.243900 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75c11d-8bad-4d95-a2af-6eef672160d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:51.244638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75c11d-8bad-4d95-a2af-6eef672160d1 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:51.244921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75c11d-8bad-4d95-a2af-6eef672160d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:51.245193 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75c11d-8bad-4d95-a2af-6eef672160d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.247392 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75c11d-8bad-4d95-a2af-6eef672160d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:51.247821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75c11d-8bad-4d95-a2af-6eef672160d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:51.249229 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75c11d-8bad-4d95-a2af-6eef672160d1 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:51.268952 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 298/579] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:51.268952 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75c11d-8bad-4d95-a2af-6eef672160d1 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:51.303835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:51.312704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee2a35-b156-4efe-a14b-dab1b93ff137 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:51.347718 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a01ccf-21dc-4e6c-a3df-7df262f9ea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:51.348413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a01ccf-21dc-4e6c-a3df-7df262f9ea3e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:51.348700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a01ccf-21dc-4e6c-a3df-7df262f9ea3e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:51.348997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a01ccf-21dc-4e6c-a3df-7df262f9ea3e None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.349835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a01ccf-21dc-4e6c-a3df-7df262f9ea3e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:51.350134 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a01ccf-21dc-4e6c-a3df-7df262f9ea3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:51.352605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a01ccf-21dc-4e6c-a3df-7df262f9ea3e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:51.371398 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 282/580] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jul 5 09:59:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:51.372289 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a01ccf-21dc-4e6c-a3df-7df262f9ea3e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:51.381792 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75c11d-8bad-4d95-a2af-6eef672160d1 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:51.421881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf8a88-e5ce-4a9a-b558-3bf8d11796ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:51.422564 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf8a88-e5ce-4a9a-b558-3bf8d11796ef None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:51.422806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf8a88-e5ce-4a9a-b558-3bf8d11796ef None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:51.423072 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf8a88-e5ce-4a9a-b558-3bf8d11796ef None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.423821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf8a88-e5ce-4a9a-b558-3bf8d11796ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:51.424125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf8a88-e5ce-4a9a-b558-3bf8d11796ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:51.425491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf8a88-e5ce-4a9a-b558-3bf8d11796ef None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:51.438379 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 299/581] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:51.438379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf8a88-e5ce-4a9a-b558-3bf8d11796ef None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:51.474185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:51.483963 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a01ccf-21dc-4e6c-a3df-7df262f9ea3e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:51.521787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406beaa5-c0e1-458a-a080-dead1c746c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:51.522672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406beaa5-c0e1-458a-a080-dead1c746c48 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:51.523002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406beaa5-c0e1-458a-a080-dead1c746c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:51.523239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406beaa5-c0e1-458a-a080-dead1c746c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.533888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406beaa5-c0e1-458a-a080-dead1c746c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:51.535378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406beaa5-c0e1-458a-a080-dead1c746c48 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:51.549954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf8a88-e5ce-4a9a-b558-3bf8d11796ef None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-406beaa5-c0e1-458a-a080-dead1c746c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:51.553223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:51.554101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:51.555009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:51.555009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:51.555009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:51.555009 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:51.555009 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 283/582] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:51.555757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406beaa5-c0e1-458a-a080-dead1c746c48 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:51.561023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:51.590205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bec9a8-a50a-4e55-9e09-0e420fdd36ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:51.591096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bec9a8-a50a-4e55-9e09-0e420fdd36ab None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:51.591380 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bec9a8-a50a-4e55-9e09-0e420fdd36ab None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:51.591656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bec9a8-a50a-4e55-9e09-0e420fdd36ab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.598948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bec9a8-a50a-4e55-9e09-0e420fdd36ab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:51.600391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bec9a8-a50a-4e55-9e09-0e420fdd36ab None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-11bec9a8-a50a-4e55-9e09-0e420fdd36ab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:51.616053 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:51.617232 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:51.618407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:51.618407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:51.618407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:51.618407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:51.618407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:51.618407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:51.618407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:51.618407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:51.618407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 09:59:51.618407 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:51.621997 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 300/583] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:51.622852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bec9a8-a50a-4e55-9e09-0e420fdd36ab None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:51.678050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406beaa5-c0e1-458a-a080-dead1c746c48 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:51.714728 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7402b-55d3-45c6-955b-7bd9d58773a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:51.715931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7402b-55d3-45c6-955b-7bd9d58773a4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:51.716315 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7402b-55d3-45c6-955b-7bd9d58773a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:51.716675 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7402b-55d3-45c6-955b-7bd9d58773a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.719490 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:51.726821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bec9a8-a50a-4e55-9e09-0e420fdd36ab None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:51.730354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7402b-55d3-45c6-955b-7bd9d58773a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:51.731959 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7402b-55d3-45c6-955b-7bd9d58773a4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-46e7402b-55d3-45c6-955b-7bd9d58773a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:51.749349 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:51.750447 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:51.751489 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:51.751489 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:51.751489 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:51.751489 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:51.757564 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 284/584] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:51.757564 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7402b-55d3-45c6-955b-7bd9d58773a4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:51.773508 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4966f0-9047-47e1-b3fd-c4fbb5435fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:51.774471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4966f0-9047-47e1-b3fd-c4fbb5435fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:51.774845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4966f0-9047-47e1-b3fd-c4fbb5435fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:51.775300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4966f0-9047-47e1-b3fd-c4fbb5435fe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.787716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4966f0-9047-47e1-b3fd-c4fbb5435fe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:51.789289 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4966f0-9047-47e1-b3fd-c4fbb5435fe9 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-8e4966f0-9047-47e1-b3fd-c4fbb5435fe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:51.810124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:51.811254 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:51.812345 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:51.812345 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:51.812345 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:51.812345 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:51.817979 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 301/585] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:51.818127 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:51.818245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4966f0-9047-47e1-b3fd-c4fbb5435fe9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:51.866106 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7402b-55d3-45c6-955b-7bd9d58773a4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:51.905426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d54cf-b479-49cb-b5ea-8d6c808d83a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:51.906103 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d54cf-b479-49cb-b5ea-8d6c808d83a4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:51.906421 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d54cf-b479-49cb-b5ea-8d6c808d83a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:51.906702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d54cf-b479-49cb-b5ea-8d6c808d83a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.907491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d54cf-b479-49cb-b5ea-8d6c808d83a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:51.907816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d54cf-b479-49cb-b5ea-8d6c808d83a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:51.909231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d54cf-b479-49cb-b5ea-8d6c808d83a4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:51.928241 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 285/586] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:51.928241 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d54cf-b479-49cb-b5ea-8d6c808d83a4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:51.939457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4966f0-9047-47e1-b3fd-c4fbb5435fe9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:51.979048 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734139e8-7d1e-45cf-bf4a-ef4d94a179b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:51.979731 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734139e8-7d1e-45cf-bf4a-ef4d94a179b0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:51.980011 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734139e8-7d1e-45cf-bf4a-ef4d94a179b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:51.980282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734139e8-7d1e-45cf-bf4a-ef4d94a179b0 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:51.981150 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734139e8-7d1e-45cf-bf4a-ef4d94a179b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:51.981442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734139e8-7d1e-45cf-bf4a-ef4d94a179b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:51.982911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734139e8-7d1e-45cf-bf4a-ef4d94a179b0 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.001170 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 302/587] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jul 5 09:59:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:52.006575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734139e8-7d1e-45cf-bf4a-ef4d94a179b0 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.015281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:52.035179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d54cf-b479-49cb-b5ea-8d6c808d83a4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:52.050574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:52.073648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2f8a4-b985-4c2e-b19f-c7dbe464ead9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:52.074348 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2f8a4-b985-4c2e-b19f-c7dbe464ead9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:52.074637 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2f8a4-b985-4c2e-b19f-c7dbe464ead9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:52.074912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2f8a4-b985-4c2e-b19f-c7dbe464ead9 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:52.075637 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2f8a4-b985-4c2e-b19f-c7dbe464ead9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:52.075978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2f8a4-b985-4c2e-b19f-c7dbe464ead9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:52.077344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2f8a4-b985-4c2e-b19f-c7dbe464ead9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.088987 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 286/588] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:52.089783 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2f8a4-b985-4c2e-b19f-c7dbe464ead9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.120851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734139e8-7d1e-45cf-bf4a-ef4d94a179b0 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:52.156665 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ceadd-8b1e-4566-925b-853ca0ca35ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:52.157831 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ceadd-8b1e-4566-925b-853ca0ca35ae None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:52.158508 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ceadd-8b1e-4566-925b-853ca0ca35ae None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:52.158926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ceadd-8b1e-4566-925b-853ca0ca35ae None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:52.160561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ceadd-8b1e-4566-925b-853ca0ca35ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:52.161063 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ceadd-8b1e-4566-925b-853ca0ca35ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:52.162625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ceadd-8b1e-4566-925b-853ca0ca35ae None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.183936 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 303/589] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:52.183936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ceadd-8b1e-4566-925b-853ca0ca35ae None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.196125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2f8a4-b985-4c2e-b19f-c7dbe464ead9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:52.232345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3af90-02a4-4629-9c77-223b1225ff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:52.233137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3af90-02a4-4629-9c77-223b1225ff53 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:52.233431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3af90-02a4-4629-9c77-223b1225ff53 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:52.233687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3af90-02a4-4629-9c77-223b1225ff53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:52.240598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3af90-02a4-4629-9c77-223b1225ff53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:52.242084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3af90-02a4-4629-9c77-223b1225ff53 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.252070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-bef3af90-02a4-4629-9c77-223b1225ff53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:52.258163 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:52.259394 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:52.260603 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:52.260603 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:52.260603 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:52.260603 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:52.260603 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:52.260603 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:52.260603 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:52.260603 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:52.260603 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 287/590] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:52.263808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3af90-02a4-4629-9c77-223b1225ff53 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.291300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ceadd-8b1e-4566-925b-853ca0ca35ae None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:52.302629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:52.328058 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76425923-6b2a-49cb-b5f8-5fd05f09edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:52.328856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76425923-6b2a-49cb-b5f8-5fd05f09edb3 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:52.329138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76425923-6b2a-49cb-b5f8-5fd05f09edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:52.329409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76425923-6b2a-49cb-b5f8-5fd05f09edb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:52.340977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76425923-6b2a-49cb-b5f8-5fd05f09edb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:52.342446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76425923-6b2a-49cb-b5f8-5fd05f09edb3 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-76425923-6b2a-49cb-b5f8-5fd05f09edb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:52.359538 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:52.360456 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:52.361385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:52.361385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:52.361385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:52.361385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:52.361385 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 304/591] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:52.361668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76425923-6b2a-49cb-b5f8-5fd05f09edb3 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.376156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3af90-02a4-4629-9c77-223b1225ff53 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:52.409235 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf5b74-1e59-4772-9880-71a04457d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:52.410042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf5b74-1e59-4772-9880-71a04457d57e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:52.410358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf5b74-1e59-4772-9880-71a04457d57e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:52.410632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf5b74-1e59-4772-9880-71a04457d57e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:52.419531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf5b74-1e59-4772-9880-71a04457d57e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:52.421007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf5b74-1e59-4772-9880-71a04457d57e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-dfdf5b74-1e59-4772-9880-71a04457d57e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:52.435164 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:52.436048 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:52.436916 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:52.436916 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:52.436916 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:52.436916 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:52.436916 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:52.436916 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:52.436916 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:52.436916 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:52.436916 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 288/592] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:52.437389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf5b74-1e59-4772-9880-71a04457d57e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.480618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76425923-6b2a-49cb-b5f8-5fd05f09edb3 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:52.500249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:52.519270 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad964e-4a73-469f-befa-8786e1d9d4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:52.520080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad964e-4a73-469f-befa-8786e1d9d4fe None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:52.520484 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad964e-4a73-469f-befa-8786e1d9d4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:52.520763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad964e-4a73-469f-befa-8786e1d9d4fe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:52.529121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad964e-4a73-469f-befa-8786e1d9d4fe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:52.530610 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad964e-4a73-469f-befa-8786e1d9d4fe None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.543314 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-e3ad964e-4a73-469f-befa-8786e1d9d4fe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:52.545358 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:52.546573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 305/593] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:52.552699 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad964e-4a73-469f-befa-8786e1d9d4fe None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.558211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf5b74-1e59-4772-9880-71a04457d57e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:52.595242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d20d4-bcc6-44fb-a7b8-fc936d328844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:52.595923 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d20d4-bcc6-44fb-a7b8-fc936d328844 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:52.596217 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d20d4-bcc6-44fb-a7b8-fc936d328844 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:52.596495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d20d4-bcc6-44fb-a7b8-fc936d328844 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:52.597286 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d20d4-bcc6-44fb-a7b8-fc936d328844 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:52.597596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d20d4-bcc6-44fb-a7b8-fc936d328844 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:52.599128 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d20d4-bcc6-44fb-a7b8-fc936d328844 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.616399 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 289/594] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:52.616399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d20d4-bcc6-44fb-a7b8-fc936d328844 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.656025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad964e-4a73-469f-befa-8786e1d9d4fe None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:52.691091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a6012-7ce0-4f42-933a-7738c00a4214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:52.691688 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a6012-7ce0-4f42-933a-7738c00a4214 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:52.691972 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a6012-7ce0-4f42-933a-7738c00a4214 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:52.692319 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a6012-7ce0-4f42-933a-7738c00a4214 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:52.693052 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a6012-7ce0-4f42-933a-7738c00a4214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:52.693362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a6012-7ce0-4f42-933a-7738c00a4214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:52.695144 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a6012-7ce0-4f42-933a-7738c00a4214 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.708773 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 306/595] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:52.709590 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a6012-7ce0-4f42-933a-7738c00a4214 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.720330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d20d4-bcc6-44fb-a7b8-fc936d328844 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:52.757033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a6293-0a68-44ee-a774-6e8f492475a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:52.757699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a6293-0a68-44ee-a774-6e8f492475a8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:52.757984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a6293-0a68-44ee-a774-6e8f492475a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:52.758281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a6293-0a68-44ee-a774-6e8f492475a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:52.759088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a6293-0a68-44ee-a774-6e8f492475a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:52.759478 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a6293-0a68-44ee-a774-6e8f492475a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:52.760851 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a6293-0a68-44ee-a774-6e8f492475a8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.766911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:52.779045 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 290/596] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:52.779045 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a6293-0a68-44ee-a774-6e8f492475a8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.791243 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:52.831640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a6012-7ce0-4f42-933a-7738c00a4214 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:52.865583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48effc-96f7-48c0-b387-d2fb0fafcbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:52.866274 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48effc-96f7-48c0-b387-d2fb0fafcbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:52.866560 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48effc-96f7-48c0-b387-d2fb0fafcbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:52.866831 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48effc-96f7-48c0-b387-d2fb0fafcbe8 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:52.867661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48effc-96f7-48c0-b387-d2fb0fafcbe8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:52.867969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48effc-96f7-48c0-b387-d2fb0fafcbe8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:52.869391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48effc-96f7-48c0-b387-d2fb0fafcbe8 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.875346 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a6293-0a68-44ee-a774-6e8f492475a8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:52.885807 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 307/597] 10.222.0.16 () {56 vars in 1139 bytes} [Tue Jul 5 09:59:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:52.886460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48effc-96f7-48c0-b387-d2fb0fafcbe8 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.909970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77106a-1fb6-4b1a-b339-3c3a20846e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:52.911046 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77106a-1fb6-4b1a-b339-3c3a20846e02 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:52.911339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77106a-1fb6-4b1a-b339-3c3a20846e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:52.911617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77106a-1fb6-4b1a-b339-3c3a20846e02 None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/fee6ee8f99c34aa281bf02ecd3f23216/roles/1821d382b71242ff8f94e4930b6ab71d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:52.925760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77106a-1fb6-4b1a-b339-3c3a20846e02 None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=fee6ee8f99c34aa281bf02ecd3f23216, role_id=1821d382b71242ff8f94e4930b6ab71d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:52.927378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77106a-1fb6-4b1a-b339-3c3a20846e02 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:52.940446 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 291/598] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:59:52 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/fee6ee8f99c34aa281bf02ecd3f23216/roles/1821d382b71242ff8f94e4930b6ab71d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:52.941322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77106a-1fb6-4b1a-b339-3c3a20846e02 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:52.985067 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:52.991108 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48effc-96f7-48c0-b387-d2fb0fafcbe8 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:53.030453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:53.032215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659657c-2e46-4931-bb65-31971755c27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:53.033082 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659657c-2e46-4931-bb65-31971755c27e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:53.033386 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659657c-2e46-4931-bb65-31971755c27e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:53.033667 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659657c-2e46-4931-bb65-31971755c27e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:53.041342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659657c-2e46-4931-bb65-31971755c27e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:53.042985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659657c-2e46-4931-bb65-31971755c27e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-f659657c-2e46-4931-bb65-31971755c27e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:53.057787 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:53.059004 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:53.065942 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:53.065942 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:53.065942 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:53.065942 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:53.065942 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:53.065942 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:53.065942 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:53.065942 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:53.065942 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 308/599] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:53.065942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56a773-43fb-4085-9b39-51f484a2919e None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:53.065942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56a773-43fb-4085-9b39-51f484a2919e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:53.067269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56a773-43fb-4085-9b39-51f484a2919e None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:53.067269 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 309/600] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:53.068071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56a773-43fb-4085-9b39-51f484a2919e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:53.079384 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77106a-1fb6-4b1a-b339-3c3a20846e02 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:53.120827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a5858-4ef6-4679-9db7-8e09ca6c00af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:53.121766 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a5858-4ef6-4679-9db7-8e09ca6c00af None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:53.122054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a5858-4ef6-4679-9db7-8e09ca6c00af None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:53.122362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a5858-4ef6-4679-9db7-8e09ca6c00af None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/b97736e49e2a4bfb9cf7093cfa8b2b16/roles/1821d382b71242ff8f94e4930b6ab71d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:53.137753 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a5858-4ef6-4679-9db7-8e09ca6c00af None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=b97736e49e2a4bfb9cf7093cfa8b2b16, role_id=1821d382b71242ff8f94e4930b6ab71d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:53.139313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a5858-4ef6-4679-9db7-8e09ca6c00af None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:53.154339 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 292/601] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:59:52 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/b97736e49e2a4bfb9cf7093cfa8b2b16/roles/1821d382b71242ff8f94e4930b6ab71d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:53.154339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a5858-4ef6-4679-9db7-8e09ca6c00af None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:53.198205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56a773-43fb-4085-9b39-51f484a2919e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:53.245650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6a3a9-6f90-4a9c-b579-de4ec16b8614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:53.246852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a3a9-6f90-4a9c-b579-de4ec16b8614 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:53.247334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a3a9-6f90-4a9c-b579-de4ec16b8614 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:53.247807 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a3a9-6f90-4a9c-b579-de4ec16b8614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:53.260415 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6a3a9-6f90-4a9c-b579-de4ec16b8614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:53.262208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6a3a9-6f90-4a9c-b579-de4ec16b8614 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-cea6a3a9-6f90-4a9c-b579-de4ec16b8614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:53.281397 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:53.282578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:53.283686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:53.283686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:53.283686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:53.283686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:53.283907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a5858-4ef6-4679-9db7-8e09ca6c00af None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:53.290199 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 310/602] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:53.292052 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548acb8-4d9e-4511-92ca-0b6ad16ecdc9 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:53.292501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548acb8-4d9e-4511-92ca-0b6ad16ecdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:53.292968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548acb8-4d9e-4511-92ca-0b6ad16ecdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:53.300151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:53.311759 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a548acb8-4d9e-4511-92ca-0b6ad16ecdc9 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:53.313220 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a548acb8-4d9e-4511-92ca-0b6ad16ecdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:53.315383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:53.324111 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05dde1-7b02-4b8a-94bd-cdf474e7200a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:53.324793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05dde1-7b02-4b8a-94bd-cdf474e7200a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:53.325080 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05dde1-7b02-4b8a-94bd-cdf474e7200a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:53.325358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05dde1-7b02-4b8a-94bd-cdf474e7200a None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:53.326160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05dde1-7b02-4b8a-94bd-cdf474e7200a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:53.326508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05dde1-7b02-4b8a-94bd-cdf474e7200a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:53.327976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05dde1-7b02-4b8a-94bd-cdf474e7200a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:53.341016 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 293/603] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:53.341866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05dde1-7b02-4b8a-94bd-cdf474e7200a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:53.389715 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 311/604] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:53.390772 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548acb8-4d9e-4511-92ca-0b6ad16ecdc9 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:53.451712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05dde1-7b02-4b8a-94bd-cdf474e7200a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:53.488448 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b5df1-8c0c-4dba-abd6-47df976826bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:53.490125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b5df1-8c0c-4dba-abd6-47df976826bc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:53.490787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b5df1-8c0c-4dba-abd6-47df976826bc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:53.491364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b5df1-8c0c-4dba-abd6-47df976826bc None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:53.492462 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b5df1-8c0c-4dba-abd6-47df976826bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:53.493252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b5df1-8c0c-4dba-abd6-47df976826bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:53.494870 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b5df1-8c0c-4dba-abd6-47df976826bc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:53.507823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548acb8-4d9e-4511-92ca-0b6ad16ecdc9 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:53.508776 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 294/605] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:53.509928 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b5df1-8c0c-4dba-abd6-47df976826bc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:53.548358 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e98bf-1c74-4c57-85e1-f0e2c8672d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:53.549434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e98bf-1c74-4c57-85e1-f0e2c8672d74 None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:53.549574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e98bf-1c74-4c57-85e1-f0e2c8672d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:53.549842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e98bf-1c74-4c57-85e1-f0e2c8672d74 None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/dc11037d649241608ea1378cf4692dad/roles/1821d382b71242ff8f94e4930b6ab71d` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:53.556048 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:53.559816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:53.565655 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e98bf-1c74-4c57-85e1-f0e2c8672d74 None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=dc11037d649241608ea1378cf4692dad, role_id=1821d382b71242ff8f94e4930b6ab71d)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:53.567260 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e98bf-1c74-4c57-85e1-f0e2c8672d74 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:53.584426 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 312/606] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:59:53 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/dc11037d649241608ea1378cf4692dad/roles/1821d382b71242ff8f94e4930b6ab71d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:53.585449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e98bf-1c74-4c57-85e1-f0e2c8672d74 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:53.671948 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b5df1-8c0c-4dba-abd6-47df976826bc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:53.738727 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6364c-0403-498d-97bc-ba3f78f9369f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:53.739553 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6364c-0403-498d-97bc-ba3f78f9369f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:53.739836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6364c-0403-498d-97bc-ba3f78f9369f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:53.740115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6364c-0403-498d-97bc-ba3f78f9369f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:53.765771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6364c-0403-498d-97bc-ba3f78f9369f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-fce6364c-0403-498d-97bc-ba3f78f9369f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:53.773844 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:53.774833 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:53.775672 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:53.778417 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 295/607] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:53.784108 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6364c-0403-498d-97bc-ba3f78f9369f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:53.788184 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e98bf-1c74-4c57-85e1-f0e2c8672d74 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:53.831445 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac9a0f-d828-46a7-8639-b8a4a295276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:53.833625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac9a0f-d828-46a7-8639-b8a4a295276e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:53.834110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac9a0f-d828-46a7-8639-b8a4a295276e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:53.835202 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac9a0f-d828-46a7-8639-b8a4a295276e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:53.844697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac9a0f-d828-46a7-8639-b8a4a295276e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:53.846232 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac9a0f-d828-46a7-8639-b8a4a295276e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-feac9a0f-d828-46a7-8639-b8a4a295276e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:53.862571 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:53.863468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:53.864951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:53.864951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:53.864951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:53.864951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:53.864951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:53.864951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:53.864951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:53.864951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:53.864951 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 313/608] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:53.869605 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac9a0f-d828-46a7-8639-b8a4a295276e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:53.897771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:53.906282 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6364c-0403-498d-97bc-ba3f78f9369f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:53.942625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c552c-1118-4105-95ea-90a1f6b0dbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:53.943751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c552c-1118-4105-95ea-90a1f6b0dbda None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:53.944232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c552c-1118-4105-95ea-90a1f6b0dbda None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:53.944709 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c552c-1118-4105-95ea-90a1f6b0dbda None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:53.945751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c552c-1118-4105-95ea-90a1f6b0dbda None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:53.946326 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c552c-1118-4105-95ea-90a1f6b0dbda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:53.947905 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c552c-1118-4105-95ea-90a1f6b0dbda None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:53.957224 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 296/609] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:53.964322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c552c-1118-4105-95ea-90a1f6b0dbda None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:53.979928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:54.106459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac9a0f-d828-46a7-8639-b8a4a295276e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:54.128768 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c552c-1118-4105-95ea-90a1f6b0dbda None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:54.163466 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f7483-6f22-47ca-a2f5-c67a66118e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:54.164442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7483-6f22-47ca-a2f5-c67a66118e28 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:54.164883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7483-6f22-47ca-a2f5-c67a66118e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:54.165309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7483-6f22-47ca-a2f5-c67a66118e28 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:54.166300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f7483-6f22-47ca-a2f5-c67a66118e28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:54.167734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f7483-6f22-47ca-a2f5-c67a66118e28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:54.169343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f7483-6f22-47ca-a2f5-c67a66118e28 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:54.182538 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 314/610] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 09:59:54.188180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f7483-6f22-47ca-a2f5-c67a66118e28 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:54.203613 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fb8ed-f651-4fe0-bd25-1ca9f10e6d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:54.204665 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fb8ed-f651-4fe0-bd25-1ca9f10e6d09 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:54.207222 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fb8ed-f651-4fe0-bd25-1ca9f10e6d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:54.208132 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fb8ed-f651-4fe0-bd25-1ca9f10e6d09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:54.224062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fb8ed-f651-4fe0-bd25-1ca9f10e6d09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:54.225763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fb8ed-f651-4fe0-bd25-1ca9f10e6d09 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-158fb8ed-f651-4fe0-bd25-1ca9f10e6d09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:54.243560 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:54.244497 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:54.245407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:54.245407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:54.245407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:54.245407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:54.245407 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 297/611] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:54.252939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fb8ed-f651-4fe0-bd25-1ca9f10e6d09 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:54.268134 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:54.366905 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fb8ed-f651-4fe0-bd25-1ca9f10e6d09 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:54.405394 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d936c-3ae9-4b6e-8a12-2edbd2633321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:54.406086 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d936c-3ae9-4b6e-8a12-2edbd2633321 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:54.406400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d936c-3ae9-4b6e-8a12-2edbd2633321 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:54.406681 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d936c-3ae9-4b6e-8a12-2edbd2633321 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:54.407416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d936c-3ae9-4b6e-8a12-2edbd2633321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:54.407730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d936c-3ae9-4b6e-8a12-2edbd2633321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:54.410524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d936c-3ae9-4b6e-8a12-2edbd2633321 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:54.423512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f7483-6f22-47ca-a2f5-c67a66118e28 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:54.440195 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 298/612] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:54.443321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:54.447284 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d936c-3ae9-4b6e-8a12-2edbd2633321 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:54.507104 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802d214-655d-495b-a1a7-95d9fb3828ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:54.508511 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802d214-655d-495b-a1a7-95d9fb3828ab None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:54.509061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802d214-655d-495b-a1a7-95d9fb3828ab None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:54.509583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802d214-655d-495b-a1a7-95d9fb3828ab None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/e0865f21da0a4c95a84c0099a1b9a78b/roles/1821d382b71242ff8f94e4930b6ab71d` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:54.541236 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802d214-655d-495b-a1a7-95d9fb3828ab None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=e0865f21da0a4c95a84c0099a1b9a78b, role_id=1821d382b71242ff8f94e4930b6ab71d)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:54.543169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802d214-655d-495b-a1a7-95d9fb3828ab None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:54.545406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:54.564626 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 315/613] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:59:54 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/e0865f21da0a4c95a84c0099a1b9a78b/roles/1821d382b71242ff8f94e4930b6ab71d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:54.587746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d936c-3ae9-4b6e-8a12-2edbd2633321 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:54.627311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bfdb1-dcb5-4e17-a3f0-772fadc14939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:54.628517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bfdb1-dcb5-4e17-a3f0-772fadc14939 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:54.629004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bfdb1-dcb5-4e17-a3f0-772fadc14939 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:54.629493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bfdb1-dcb5-4e17-a3f0-772fadc14939 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:54.638440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599bfdb1-dcb5-4e17-a3f0-772fadc14939 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:54.640260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599bfdb1-dcb5-4e17-a3f0-772fadc14939 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-599bfdb1-dcb5-4e17-a3f0-772fadc14939 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:54.656313 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:54.657328 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:54.658281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:54.658281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:54.658281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:54.658281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:54.658281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:54.658281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:54.658281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:54.658281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:54.658281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 09:59:54.658281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:54.661080 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 299/614] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 09:59:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:54.666731 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802d214-655d-495b-a1a7-95d9fb3828ab None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:54.823089 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802d214-655d-495b-a1a7-95d9fb3828ab None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:54.860000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23f336-9a42-46f8-899a-6cec8f4381e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:54.860686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23f336-9a42-46f8-899a-6cec8f4381e9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:54.860966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23f336-9a42-46f8-899a-6cec8f4381e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:54.861241 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23f336-9a42-46f8-899a-6cec8f4381e9 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:54.862077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23f336-9a42-46f8-899a-6cec8f4381e9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:54.862410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23f336-9a42-46f8-899a-6cec8f4381e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:54.863862 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23f336-9a42-46f8-899a-6cec8f4381e9 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:54.882656 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 316/615] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jul 5 09:59:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:54.887759 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bfdb1-dcb5-4e17-a3f0-772fadc14939 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:55.026147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bfdb1-dcb5-4e17-a3f0-772fadc14939 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:55.063650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:55.073308 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a283214-3ab2-4621-bdcd-1278ce84131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:55.074128 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a283214-3ab2-4621-bdcd-1278ce84131e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:55.074434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a283214-3ab2-4621-bdcd-1278ce84131e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:55.074710 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a283214-3ab2-4621-bdcd-1278ce84131e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:55.095115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a283214-3ab2-4621-bdcd-1278ce84131e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:55.096583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a283214-3ab2-4621-bdcd-1278ce84131e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9a283214-3ab2-4621-bdcd-1278ce84131e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:55.117774 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:55.118753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:55.119682 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:55.119682 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:55.119682 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:55.119682 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:55.119682 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 300/616] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:55.124864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23f336-9a42-46f8-899a-6cec8f4381e9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:55.149986 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:55.263897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb15fd-cd44-4cc7-91ec-5656f515aee4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:55.264247 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb15fd-cd44-4cc7-91ec-5656f515aee4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:55.264454 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb15fd-cd44-4cc7-91ec-5656f515aee4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:55.265054 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23f336-9a42-46f8-899a-6cec8f4381e9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:55.266086 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 301/617] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:55.271626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd101-966b-4db7-a164-a6a91393b002 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:55.271915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd101-966b-4db7-a164-a6a91393b002 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:55.272199 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd101-966b-4db7-a164-a6a91393b002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:55.300028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b814f10-b81a-4a24-823c-b6b81dc6a68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:55.300624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b814f10-b81a-4a24-823c-b6b81dc6a68e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:55.300907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b814f10-b81a-4a24-823c-b6b81dc6a68e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:55.301185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b814f10-b81a-4a24-823c-b6b81dc6a68e None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:55.302079 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b814f10-b81a-4a24-823c-b6b81dc6a68e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:55.302823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b814f10-b81a-4a24-823c-b6b81dc6a68e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:55.303831 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b814f10-b81a-4a24-823c-b6b81dc6a68e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:55.315257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e2cbd101-966b-4db7-a164-a6a91393b002 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:55.315963 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 317/618] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:55.317113 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e2cbd101-966b-4db7-a164-a6a91393b002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:55.323979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b814f10-b81a-4a24-823c-b6b81dc6a68e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:55.423843 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 302/619] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:55.455467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd101-966b-4db7-a164-a6a91393b002 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:55.462876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:55.489488 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b814f10-b81a-4a24-823c-b6b81dc6a68e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:55.533010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0181ef-50a3-4379-84db-d11d2b47730f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:55.533968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0181ef-50a3-4379-84db-d11d2b47730f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:55.534271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0181ef-50a3-4379-84db-d11d2b47730f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:55.534853 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0181ef-50a3-4379-84db-d11d2b47730f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:55.543430 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0181ef-50a3-4379-84db-d11d2b47730f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:55.545638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0181ef-50a3-4379-84db-d11d2b47730f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-ad0181ef-50a3-4379-84db-d11d2b47730f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:55.563386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:55.564510 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:55.566379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:55.566379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:55.566379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:55.566379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:55.566379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:55.566379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:55.566379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:55.566379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:55.566379 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 318/620] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:55.566379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf30a1-ab80-46e0-96bb-172a4657e053 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:55.567204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf30a1-ab80-46e0-96bb-172a4657e053 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:55.567204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf30a1-ab80-46e0-96bb-172a4657e053 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:55.568832 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 319/621] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:55.573899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5f714-7aca-4126-a586-049c19e319d2 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:55.574189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5f714-7aca-4126-a586-049c19e319d2 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:55.574499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5f714-7aca-4126-a586-049c19e319d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:55.590280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd101-966b-4db7-a164-a6a91393b002 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:55.599394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-01b5f714-7aca-4126-a586-049c19e319d2 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:55.601171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-01b5f714-7aca-4126-a586-049c19e319d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:55.602729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:55.633286 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a859003-fdf4-4487-947d-aa9df8cd9c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:55.634285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a859003-fdf4-4487-947d-aa9df8cd9c49 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:55.634578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a859003-fdf4-4487-947d-aa9df8cd9c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:55.634861 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a859003-fdf4-4487-947d-aa9df8cd9c49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:55.650908 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a859003-fdf4-4487-947d-aa9df8cd9c49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9a859003-fdf4-4487-947d-aa9df8cd9c49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:55.660080 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:55.661105 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:55.661983 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 303/622] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 09:59:55.662748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a859003-fdf4-4487-947d-aa9df8cd9c49 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:55.710437 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 320/623] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 09:59:55.710437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5f714-7aca-4126-a586-049c19e319d2 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:55.792338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a859003-fdf4-4487-947d-aa9df8cd9c49 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:55.800439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:55.828856 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce4073-bf9a-4eee-a9f7-1b3750717423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:55.829685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4073-bf9a-4eee-a9f7-1b3750717423 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:55.830069 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4073-bf9a-4eee-a9f7-1b3750717423 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:55.830490 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4073-bf9a-4eee-a9f7-1b3750717423 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:55.831489 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce4073-bf9a-4eee-a9f7-1b3750717423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:55.831911 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce4073-bf9a-4eee-a9f7-1b3750717423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:55.833865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce4073-bf9a-4eee-a9f7-1b3750717423 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:55.851634 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 304/624] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:55.851634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce4073-bf9a-4eee-a9f7-1b3750717423 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:55.851874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5f714-7aca-4126-a586-049c19e319d2 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:55.884391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:55.897665 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33ea44-6fe3-453e-855e-790453ced138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:55.899179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33ea44-6fe3-453e-855e-790453ced138 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:55.899697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33ea44-6fe3-453e-855e-790453ced138 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:55.900570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33ea44-6fe3-453e-855e-790453ced138 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:55.901791 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33ea44-6fe3-453e-855e-790453ced138 None admin] RBAC: Adding query filter params (name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:55.902566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33ea44-6fe3-453e-855e-790453ced138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:55.904897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33ea44-6fe3-453e-855e-790453ced138 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:55.916999 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 321/625] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:55.918413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33ea44-6fe3-453e-855e-790453ced138 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:55.966682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce4073-bf9a-4eee-a9f7-1b3750717423 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:56.016064 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768321da-1043-41d0-96a9-bf7955e6fa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:56.016872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768321da-1043-41d0-96a9-bf7955e6fa2a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.017154 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768321da-1043-41d0-96a9-bf7955e6fa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:56.017425 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768321da-1043-41d0-96a9-bf7955e6fa2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:56.032113 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768321da-1043-41d0-96a9-bf7955e6fa2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-768321da-1043-41d0-96a9-bf7955e6fa2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:56.043224 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:56.044180 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:56.045040 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 305/626] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:56.045740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768321da-1043-41d0-96a9-bf7955e6fa2a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:56.050874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33ea44-6fe3-453e-855e-790453ced138 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:56.080176 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:56.088892 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf84bf-47d4-40fe-942e-5a72957a2d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:56.089804 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf84bf-47d4-40fe-942e-5a72957a2d16 None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.090089 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf84bf-47d4-40fe-942e-5a72957a2d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:56.090392 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf84bf-47d4-40fe-942e-5a72957a2d16 None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/dbfded29b70140a387f7b8c07c296861/roles/1821d382b71242ff8f94e4930b6ab71d` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:56.105630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf84bf-47d4-40fe-942e-5a72957a2d16 None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=dbfded29b70140a387f7b8c07c296861, role_id=1821d382b71242ff8f94e4930b6ab71d)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:56.107126 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf84bf-47d4-40fe-942e-5a72957a2d16 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:56.120440 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 322/627] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 09:59:55 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/dbfded29b70140a387f7b8c07c296861/roles/1821d382b71242ff8f94e4930b6ab71d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 09:59:56.121224 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf84bf-47d4-40fe-942e-5a72957a2d16 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:56.154686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:56.165667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768321da-1043-41d0-96a9-bf7955e6fa2a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:56.203690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcaf9c-bec5-41d4-8b72-e3d2ff2e85d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:56.204512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcaf9c-bec5-41d4-8b72-e3d2ff2e85d9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.204799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcaf9c-bec5-41d4-8b72-e3d2ff2e85d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:56.205076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcaf9c-bec5-41d4-8b72-e3d2ff2e85d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:56.218645 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcaf9c-bec5-41d4-8b72-e3d2ff2e85d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:56.220329 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcaf9c-bec5-41d4-8b72-e3d2ff2e85d9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-bafcaf9c-bec5-41d4-8b72-e3d2ff2e85d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:56.240227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:56.241208 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:56.242171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:56.242171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:56.242171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:56.242171 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:56.246457 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 306/628] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:56.247974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d438a4f-dd03-464c-a539-9c984a5d72e0 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.248267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d438a4f-dd03-464c-a539-9c984a5d72e0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:56.248549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d438a4f-dd03-464c-a539-9c984a5d72e0 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:56.254353 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 307/629] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:56.255333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d438a4f-dd03-464c-a539-9c984a5d72e0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:56.270286 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf84bf-47d4-40fe-942e-5a72957a2d16 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:56.314565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a07145-0e8b-4ffd-9840-932451fcab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:56.315275 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a07145-0e8b-4ffd-9840-932451fcab72 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.315555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a07145-0e8b-4ffd-9840-932451fcab72 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:56.315830 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a07145-0e8b-4ffd-9840-932451fcab72 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:56.316603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a07145-0e8b-4ffd-9840-932451fcab72 None admin] RBAC: Adding query filter params (name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:56.316931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a07145-0e8b-4ffd-9840-932451fcab72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:56.319671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a07145-0e8b-4ffd-9840-932451fcab72 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:56.330605 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 323/630] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:56.333332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc52f06-21f9-4197-aa13-89f1a4ff9c5c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.334035 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc52f06-21f9-4197-aa13-89f1a4ff9c5c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:56.334542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc52f06-21f9-4197-aa13-89f1a4ff9c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:56.356244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-4dc52f06-21f9-4197-aa13-89f1a4ff9c5c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:56.358618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4dc52f06-21f9-4197-aa13-89f1a4ff9c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:56.385227 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d438a4f-dd03-464c-a539-9c984a5d72e0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:56.403721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:56.429267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fb31f-e583-4064-badf-709715b712f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:56.430070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb31f-e583-4064-badf-709715b712f1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.430450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb31f-e583-4064-badf-709715b712f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:56.430797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb31f-e583-4064-badf-709715b712f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:56.432140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fb31f-e583-4064-badf-709715b712f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:56.432532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fb31f-e583-4064-badf-709715b712f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:56.435146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fb31f-e583-4064-badf-709715b712f1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:56.448029 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 308/631] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:56.449234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fb31f-e583-4064-badf-709715b712f1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:56.453740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:56.466435 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 324/632] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:56.467355 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc52f06-21f9-4197-aa13-89f1a4ff9c5c None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:56.556504 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fb31f-e583-4064-badf-709715b712f1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:56.596933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54954b71-2067-49d3-965a-d9a073d6fc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:56.598647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54954b71-2067-49d3-965a-d9a073d6fc11 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.599161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54954b71-2067-49d3-965a-d9a073d6fc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:56.599625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54954b71-2067-49d3-965a-d9a073d6fc11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:56.604359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc52f06-21f9-4197-aa13-89f1a4ff9c5c None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:56.611410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54954b71-2067-49d3-965a-d9a073d6fc11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:56.613195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54954b71-2067-49d3-965a-d9a073d6fc11 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-54954b71-2067-49d3-965a-d9a073d6fc11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:56.632294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:56.633226 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:56.634111 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:56.634111 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:56.634111 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:56.634111 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:56.636399 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 309/633] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:56.637381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54954b71-2067-49d3-965a-d9a073d6fc11 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:56.648363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9452bc1-96b1-4083-bb69-721d2a16e1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:56.649876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9452bc1-96b1-4083-bb69-721d2a16e1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.650439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9452bc1-96b1-4083-bb69-721d2a16e1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:56.651240 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9452bc1-96b1-4083-bb69-721d2a16e1a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:56.660721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9452bc1-96b1-4083-bb69-721d2a16e1a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:56.663083 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9452bc1-96b1-4083-bb69-721d2a16e1a9 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-c9452bc1-96b1-4083-bb69-721d2a16e1a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:56.681802 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:56.683275 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:56.684245 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:56.684245 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:56.684245 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:56.684245 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:56.684245 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:56.684245 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:56.684245 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:56.684245 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:56.684245 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 09:59:56.684245 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:56.687055 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 325/634] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jul 5 09:59:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:56.687982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9452bc1-96b1-4083-bb69-721d2a16e1a9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:56.705344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:56.726288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:56.769169 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54954b71-2067-49d3-965a-d9a073d6fc11 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:56.815640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9452bc1-96b1-4083-bb69-721d2a16e1a9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:56.816845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1a957-fe9e-48c5-84c0-07705476786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:56.817874 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a957-fe9e-48c5-84c0-07705476786e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.818544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a957-fe9e-48c5-84c0-07705476786e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:56.818922 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a957-fe9e-48c5-84c0-07705476786e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:56.833002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1a957-fe9e-48c5-84c0-07705476786e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-60b1a957-fe9e-48c5-84c0-07705476786e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:56.840730 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:56.841705 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:56.842926 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:56.847267 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 310/635] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:56.847950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1a957-fe9e-48c5-84c0-07705476786e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:56.865726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d7425-a14c-452f-adf9-e28a6749b704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:56.867225 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d7425-a14c-452f-adf9-e28a6749b704 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.867507 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d7425-a14c-452f-adf9-e28a6749b704 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:56.867780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d7425-a14c-452f-adf9-e28a6749b704 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:56.868506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d7425-a14c-452f-adf9-e28a6749b704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:56.868815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d7425-a14c-452f-adf9-e28a6749b704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:56.870185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d7425-a14c-452f-adf9-e28a6749b704 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:56.889357 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 326/636] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:56.891787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d7425-a14c-452f-adf9-e28a6749b704 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:56.961470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1a957-fe9e-48c5-84c0-07705476786e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:56.977685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:56.996529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:56.998939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4b68a-b8f5-48e9-9584-32b1624ae723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:56.999638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4b68a-b8f5-48e9-9584-32b1624ae723 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:56.999952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4b68a-b8f5-48e9-9584-32b1624ae723 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.000227 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4b68a-b8f5-48e9-9584-32b1624ae723 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.001081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4b68a-b8f5-48e9-9584-32b1624ae723 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:57.001391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4b68a-b8f5-48e9-9584-32b1624ae723 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:57.002896 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4b68a-b8f5-48e9-9584-32b1624ae723 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:57.008500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d7425-a14c-452f-adf9-e28a6749b704 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:57.025785 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 311/637] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jul 5 09:59:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:57.025785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4b68a-b8f5-48e9-9584-32b1624ae723 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:57.048050 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d24002-cc5e-48e0-9a69-fce9cd2b752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:57.048807 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d24002-cc5e-48e0-9a69-fce9cd2b752c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:57.049222 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d24002-cc5e-48e0-9a69-fce9cd2b752c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.049451 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d24002-cc5e-48e0-9a69-fce9cd2b752c None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.050165 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d24002-cc5e-48e0-9a69-fce9cd2b752c None admin] RBAC: Adding query filter params (name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:57.050499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d24002-cc5e-48e0-9a69-fce9cd2b752c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:57.051901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d24002-cc5e-48e0-9a69-fce9cd2b752c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:57.059919 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 327/638] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:57.065341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833e35b-9e50-477e-8ffe-dded80615e40 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:57.065631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833e35b-9e50-477e-8ffe-dded80615e40 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.065961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833e35b-9e50-477e-8ffe-dded80615e40 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.070278 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 328/639] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:57.070278 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833e35b-9e50-477e-8ffe-dded80615e40 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:57.126864 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4b68a-b8f5-48e9-9584-32b1624ae723 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:57.164350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec8e5-1c46-4d4f-8b75-0115f30e14e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:57.165256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec8e5-1c46-4d4f-8b75-0115f30e14e6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:57.165646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec8e5-1c46-4d4f-8b75-0115f30e14e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.165921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec8e5-1c46-4d4f-8b75-0115f30e14e6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.172668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833e35b-9e50-477e-8ffe-dded80615e40 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:57.173597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ec8e5-1c46-4d4f-8b75-0115f30e14e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:57.175338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ec8e5-1c46-4d4f-8b75-0115f30e14e6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-823ec8e5-1c46-4d4f-8b75-0115f30e14e6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:57.194890 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:57.195873 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:57.196857 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:57.196857 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:57.196857 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:57.196857 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:57.196857 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:57.196857 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:57.196857 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:57.196857 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:57.196857 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 05 09:59:57.196857 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:57.200929 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 312/640] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:57.201806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec8e5-1c46-4d4f-8b75-0115f30e14e6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:57.213874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f88a27-7b35-4989-ac5c-02a2254206a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:57.214577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f88a27-7b35-4989-ac5c-02a2254206a5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:57.215170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f88a27-7b35-4989-ac5c-02a2254206a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.215450 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f88a27-7b35-4989-ac5c-02a2254206a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.236656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:57.239129 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:57.291089 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f88a27-7b35-4989-ac5c-02a2254206a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:57.292609 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f88a27-7b35-4989-ac5c-02a2254206a5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:57.313371 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec8e5-1c46-4d4f-8b75-0115f30e14e6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:57.348557 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391827a-c42c-41e4-9937-1591afddb3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:57.349278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391827a-c42c-41e4-9937-1591afddb3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:57.349562 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391827a-c42c-41e4-9937-1591afddb3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.349839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391827a-c42c-41e4-9937-1591afddb3d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.361552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391827a-c42c-41e4-9937-1591afddb3d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:57.363179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391827a-c42c-41e4-9937-1591afddb3d8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9391827a-c42c-41e4-9937-1591afddb3d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:57.383433 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:57.384449 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:57.385335 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:57.385335 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:57.385335 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:57.385335 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:57.389376 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 313/641] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:57.390694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f91d8-2ed7-4919-8662-40ac71a62196 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:57.390983 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f91d8-2ed7-4919-8662-40ac71a62196 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.391262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f91d8-2ed7-4919-8662-40ac71a62196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.403836 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 329/642] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:57.405153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f88a27-7b35-4989-ac5c-02a2254206a5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:57.408440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b62f91d8-2ed7-4919-8662-40ac71a62196 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:57.409738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b62f91d8-2ed7-4919-8662-40ac71a62196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:57.468365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:57.503717 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 314/643] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:57.504947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f91d8-2ed7-4919-8662-40ac71a62196 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:57.519890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f88a27-7b35-4989-ac5c-02a2254206a5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:57.526483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:57.563711 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4b796-73b5-4b6b-87a3-2e33a792a39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:57.564975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4b796-73b5-4b6b-87a3-2e33a792a39a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:57.565403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4b796-73b5-4b6b-87a3-2e33a792a39a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.565825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4b796-73b5-4b6b-87a3-2e33a792a39a None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.567168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4b796-73b5-4b6b-87a3-2e33a792a39a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:57.568114 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4b796-73b5-4b6b-87a3-2e33a792a39a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:57.570245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4b796-73b5-4b6b-87a3-2e33a792a39a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:57.595059 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 330/644] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jul 5 09:59:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:57.596110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4b796-73b5-4b6b-87a3-2e33a792a39a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:57.631565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f91d8-2ed7-4919-8662-40ac71a62196 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:57.666914 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47b70c-dfac-4006-a547-c8af175adca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:57.667637 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47b70c-dfac-4006-a547-c8af175adca1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:57.667919 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47b70c-dfac-4006-a547-c8af175adca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.668215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47b70c-dfac-4006-a547-c8af175adca1 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.668921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47b70c-dfac-4006-a547-c8af175adca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:57.669225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47b70c-dfac-4006-a547-c8af175adca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:57.670700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47b70c-dfac-4006-a547-c8af175adca1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:57.682594 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 315/645] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:57.683225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47b70c-dfac-4006-a547-c8af175adca1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:57.726670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4b796-73b5-4b6b-87a3-2e33a792a39a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:57.739745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:57.768866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdedcd6-7232-4191-a89a-05c01c203356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:57.770870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdedcd6-7232-4191-a89a-05c01c203356 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:57.772264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdedcd6-7232-4191-a89a-05c01c203356 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.772823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdedcd6-7232-4191-a89a-05c01c203356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.785212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdedcd6-7232-4191-a89a-05c01c203356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:57.787124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdedcd6-7232-4191-a89a-05c01c203356 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:57.792151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-6bdedcd6-7232-4191-a89a-05c01c203356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:57.806187 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:57.807247 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:57.808293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:57.808293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:57.808293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:57.808293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:57.814997 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 331/646] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:57.816114 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdedcd6-7232-4191-a89a-05c01c203356 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:57.824320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47b70c-dfac-4006-a547-c8af175adca1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:57.865181 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7226a-96bb-44e7-802c-023627df4fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:57.866007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7226a-96bb-44e7-802c-023627df4fed None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:57.866333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7226a-96bb-44e7-802c-023627df4fed None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.866591 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7226a-96bb-44e7-802c-023627df4fed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.877552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7226a-96bb-44e7-802c-023627df4fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-54f7226a-96bb-44e7-802c-023627df4fed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:57.885512 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:57.886510 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:57.887378 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 316/647] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:57.888071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7226a-96bb-44e7-802c-023627df4fed None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:57.936147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdedcd6-7232-4191-a89a-05c01c203356 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:57.974016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c657b4-424c-42bb-8ab9-1836a3de175c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:57.974921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c657b4-424c-42bb-8ab9-1836a3de175c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:57.975367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c657b4-424c-42bb-8ab9-1836a3de175c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:57.975798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c657b4-424c-42bb-8ab9-1836a3de175c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:57.992823 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7226a-96bb-44e7-802c-023627df4fed None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:58.007093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:58.028326 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c651435-e97c-4baa-924b-899595b0edbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:58.029138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c651435-e97c-4baa-924b-899595b0edbb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:58.029420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c651435-e97c-4baa-924b-899595b0edbb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:58.029693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c651435-e97c-4baa-924b-899595b0edbb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:58.031576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:58.038010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c651435-e97c-4baa-924b-899595b0edbb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:58.039793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c651435-e97c-4baa-924b-899595b0edbb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:58.048941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c657b4-424c-42bb-8ab9-1836a3de175c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:58.050809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c657b4-424c-42bb-8ab9-1836a3de175c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1c651435-e97c-4baa-924b-899595b0edbb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:58.053806 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:58.054754 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:58.055680 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:58.055680 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:58.055680 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:58.055680 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:58.055680 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:58.055680 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:58.055680 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:58.055680 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:58.055680 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 09:59:58.055680 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:58.055680 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 317/648] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 09:59:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:58.056268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c651435-e97c-4baa-924b-899595b0edbb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:58.159081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c651435-e97c-4baa-924b-899595b0edbb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:58.161507 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 332/649] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:58.162179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c657b4-424c-42bb-8ab9-1836a3de175c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:58.193484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761229d3-a89b-44dc-8c6a-db51f1b08ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:58.194166 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761229d3-a89b-44dc-8c6a-db51f1b08ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:58.194472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761229d3-a89b-44dc-8c6a-db51f1b08ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:58.194750 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761229d3-a89b-44dc-8c6a-db51f1b08ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:58.195473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761229d3-a89b-44dc-8c6a-db51f1b08ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:58.195763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761229d3-a89b-44dc-8c6a-db51f1b08ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:58.197143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761229d3-a89b-44dc-8c6a-db51f1b08ce5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:58.210085 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 318/650] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:58.210923 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761229d3-a89b-44dc-8c6a-db51f1b08ce5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:58.228070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:58.269171 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:58.269527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c657b4-424c-42bb-8ab9-1836a3de175c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:58.302899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56898b19-ab53-4f21-bdec-0acf9fbffd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:58.303594 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56898b19-ab53-4f21-bdec-0acf9fbffd7e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:58.303875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56898b19-ab53-4f21-bdec-0acf9fbffd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:58.304145 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56898b19-ab53-4f21-bdec-0acf9fbffd7e None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:58.304926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56898b19-ab53-4f21-bdec-0acf9fbffd7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:58.305248 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56898b19-ab53-4f21-bdec-0acf9fbffd7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:58.306674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56898b19-ab53-4f21-bdec-0acf9fbffd7e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:58.314330 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 333/651] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:58.314463 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761229d3-a89b-44dc-8c6a-db51f1b08ce5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:58.315109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56898b19-ab53-4f21-bdec-0acf9fbffd7e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:58.353216 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488baed-1a1a-49e9-9fb6-3a9c3747a908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:58.353908 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488baed-1a1a-49e9-9fb6-3a9c3747a908 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:58.354190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488baed-1a1a-49e9-9fb6-3a9c3747a908 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:58.354498 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488baed-1a1a-49e9-9fb6-3a9c3747a908 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:58.355333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488baed-1a1a-49e9-9fb6-3a9c3747a908 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:58.355642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488baed-1a1a-49e9-9fb6-3a9c3747a908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:58.357084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488baed-1a1a-49e9-9fb6-3a9c3747a908 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:58.374712 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 319/652] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jul 5 09:59:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:58.375623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488baed-1a1a-49e9-9fb6-3a9c3747a908 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:58.418321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56898b19-ab53-4f21-bdec-0acf9fbffd7e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:58.441699 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:58.454015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308614a-57b2-4ec2-8fc2-ce1afe0a9d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:58.454919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308614a-57b2-4ec2-8fc2-ce1afe0a9d09 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:58.455216 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308614a-57b2-4ec2-8fc2-ce1afe0a9d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:58.455493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308614a-57b2-4ec2-8fc2-ce1afe0a9d09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:58.466716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308614a-57b2-4ec2-8fc2-ce1afe0a9d09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:58.468233 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308614a-57b2-4ec2-8fc2-ce1afe0a9d09 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:58.475867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488baed-1a1a-49e9-9fb6-3a9c3747a908 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:58.489547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-a308614a-57b2-4ec2-8fc2-ce1afe0a9d09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:58.492157 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:58.493323 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:58.494617 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:58.494617 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:58.494617 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:58.494617 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:58.496057 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 334/653] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:58.497348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4138-1368-44b8-850b-3aa8fe76de79 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:58.497644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4138-1368-44b8-850b-3aa8fe76de79 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:58.497960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4138-1368-44b8-850b-3aa8fe76de79 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:58.499493 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 335/654] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 09:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 09:59:58.502991 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf4138-1368-44b8-850b-3aa8fe76de79 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:58.513725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d97cba-b05b-49b2-8ede-f867a4e75ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:58.514575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d97cba-b05b-49b2-8ede-f867a4e75ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:58.514847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d97cba-b05b-49b2-8ede-f867a4e75ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:58.515102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d97cba-b05b-49b2-8ede-f867a4e75ae8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:58.521833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d97cba-b05b-49b2-8ede-f867a4e75ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:58.523375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d97cba-b05b-49b2-8ede-f867a4e75ae8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-30d97cba-b05b-49b2-8ede-f867a4e75ae8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:58.537738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:58.539223 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:58.540148 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:58.540148 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:58.540148 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:58.540148 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:58.540148 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:58.540148 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:58.540148 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:58.540148 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:58.542395 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 320/655] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:58.543445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d97cba-b05b-49b2-8ede-f867a4e75ae8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:58.606861 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf4138-1368-44b8-850b-3aa8fe76de79 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:58.638841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae72ff-7b58-49b5-bc6c-30f793eda5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:58.639634 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae72ff-7b58-49b5-bc6c-30f793eda5ab None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:58.639915 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae72ff-7b58-49b5-bc6c-30f793eda5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:58.640186 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae72ff-7b58-49b5-bc6c-30f793eda5ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:58.647178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d97cba-b05b-49b2-8ede-f867a4e75ae8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:58.650605 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae72ff-7b58-49b5-bc6c-30f793eda5ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:58.652063 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae72ff-7b58-49b5-bc6c-30f793eda5ab None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-a4ae72ff-7b58-49b5-bc6c-30f793eda5ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:58.668731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:58.669785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:58.670898 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:58.670898 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:58.670898 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:58.670898 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:58.670898 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 336/656] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:58.671237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae72ff-7b58-49b5-bc6c-30f793eda5ab None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:58.682606 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1477bd-de40-4956-9f75-b4b1ba5768da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:58.683297 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1477bd-de40-4956-9f75-b4b1ba5768da None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:58.683592 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1477bd-de40-4956-9f75-b4b1ba5768da None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:58.683872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1477bd-de40-4956-9f75-b4b1ba5768da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:58.700285 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:58.725659 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:58.752159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1477bd-de40-4956-9f75-b4b1ba5768da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:58.753732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1477bd-de40-4956-9f75-b4b1ba5768da None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:58.774473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae72ff-7b58-49b5-bc6c-30f793eda5ab None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:58.808111 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35324f2-38f5-4745-86d6-ccb3cc85df8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:58.808809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35324f2-38f5-4745-86d6-ccb3cc85df8d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:58.809094 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35324f2-38f5-4745-86d6-ccb3cc85df8d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:58.809363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35324f2-38f5-4745-86d6-ccb3cc85df8d None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:58.810140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35324f2-38f5-4745-86d6-ccb3cc85df8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:58.810469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35324f2-38f5-4745-86d6-ccb3cc85df8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:58.811914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35324f2-38f5-4745-86d6-ccb3cc85df8d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:58.823584 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 337/657] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:58.824822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc6a5c-8e34-4e2b-8982-b75c30e950fa None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:58.825269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc6a5c-8e34-4e2b-8982-b75c30e950fa None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:58.825548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc6a5c-8e34-4e2b-8982-b75c30e950fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:58.843363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a0bc6a5c-8e34-4e2b-8982-b75c30e950fa None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 09:59:58.844572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a0bc6a5c-8e34-4e2b-8982-b75c30e950fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 09:59:58.855648 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 321/658] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 09:59:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 09:59:58.856527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1477bd-de40-4956-9f75-b4b1ba5768da None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:58.912683 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 338/659] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 09:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 09:59:58.913564 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc6a5c-8e34-4e2b-8982-b75c30e950fa None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:58.932177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:58.942098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:58.971654 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1477bd-de40-4956-9f75-b4b1ba5768da None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.005416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9292640-c80e-4a60-aecd-075150a39aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.006107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9292640-c80e-4a60-aecd-075150a39aaa None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.006428 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9292640-c80e-4a60-aecd-075150a39aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.006710 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9292640-c80e-4a60-aecd-075150a39aaa None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.007534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9292640-c80e-4a60-aecd-075150a39aaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:59.007845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9292640-c80e-4a60-aecd-075150a39aaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:59.009328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9292640-c80e-4a60-aecd-075150a39aaa None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:59.013042 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc6a5c-8e34-4e2b-8982-b75c30e950fa None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.020827 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 322/660] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:59.021626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9292640-c80e-4a60-aecd-075150a39aaa None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:59.051486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda5149-a4ed-4ac4-8089-c3742ddcd01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.052202 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda5149-a4ed-4ac4-8089-c3742ddcd01e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.052488 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda5149-a4ed-4ac4-8089-c3742ddcd01e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.052760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda5149-a4ed-4ac4-8089-c3742ddcd01e None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.053511 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda5149-a4ed-4ac4-8089-c3742ddcd01e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:59.053823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda5149-a4ed-4ac4-8089-c3742ddcd01e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:59.055300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda5149-a4ed-4ac4-8089-c3742ddcd01e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:59.066603 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 339/661] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:59.067656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda5149-a4ed-4ac4-8089-c3742ddcd01e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:59.125400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9292640-c80e-4a60-aecd-075150a39aaa None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.143647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:59.154622 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:59.159850 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f18760-a6c2-4dbd-a0d9-a67c6f217b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.160704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f18760-a6c2-4dbd-a0d9-a67c6f217b29 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.160983 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f18760-a6c2-4dbd-a0d9-a67c6f217b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.161253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f18760-a6c2-4dbd-a0d9-a67c6f217b29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.165790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda5149-a4ed-4ac4-8089-c3742ddcd01e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.167999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f18760-a6c2-4dbd-a0d9-a67c6f217b29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:59.169465 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f18760-a6c2-4dbd-a0d9-a67c6f217b29 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-42f18760-a6c2-4dbd-a0d9-a67c6f217b29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 09:59:59.183576 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:59.184515 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:59.185401 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 09:59:59.185401 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 09:59:59.185401 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 09:59:59.185401 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 09:59:59.185401 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 09:59:59.185401 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 09:59:59.185401 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 09:59:59.185401 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:59.185401 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 323/662] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:59.185899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f18760-a6c2-4dbd-a0d9-a67c6f217b29 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:59.205461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1ba16-aed7-48c1-9dba-10bee3993c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.206403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1ba16-aed7-48c1-9dba-10bee3993c1e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.206721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1ba16-aed7-48c1-9dba-10bee3993c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.206984 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1ba16-aed7-48c1-9dba-10bee3993c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.218166 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1ba16-aed7-48c1-9dba-10bee3993c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:59.219645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1ba16-aed7-48c1-9dba-10bee3993c1e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-7cb1ba16-aed7-48c1-9dba-10bee3993c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 09:59:59.236662 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:59.237590 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:59.238687 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 09:59:59.238687 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 09:59:59.238687 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 09:59:59.238687 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 09:59:59.238687 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 340/663] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:59.239000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1ba16-aed7-48c1-9dba-10bee3993c1e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:59.286896 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f18760-a6c2-4dbd-a0d9-a67c6f217b29 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.320099 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0abff6-1497-4879-8709-4bca3d9863c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.320925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0abff6-1497-4879-8709-4bca3d9863c6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.321216 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0abff6-1497-4879-8709-4bca3d9863c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.321491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0abff6-1497-4879-8709-4bca3d9863c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.335079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0abff6-1497-4879-8709-4bca3d9863c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-6e0abff6-1497-4879-8709-4bca3d9863c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 09:59:59.342836 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:59.343871 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 09:59:59.350106 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:59.351004 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1ba16-aed7-48c1-9dba-10bee3993c1e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.351116 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 324/664] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:59.351116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0abff6-1497-4879-8709-4bca3d9863c6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:59.379453 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:59.386479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0e7bb-e2f9-4925-9754-0cfea4b346d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.387184 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0e7bb-e2f9-4925-9754-0cfea4b346d7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.387465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0e7bb-e2f9-4925-9754-0cfea4b346d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.387738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0e7bb-e2f9-4925-9754-0cfea4b346d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.388605 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0e7bb-e2f9-4925-9754-0cfea4b346d7 None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=fee6ee8f99c34aa281bf02ecd3f23216) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:59.388914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0e7bb-e2f9-4925-9754-0cfea4b346d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:59.390493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0e7bb-e2f9-4925-9754-0cfea4b346d7 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:59.396527 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 341/665] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/role_assignments?user.id=fee6ee8f99c34aa281bf02ecd3f23216&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:59.397323 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0e7bb-e2f9-4925-9754-0cfea4b346d7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:59.398993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:59.455211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0abff6-1497-4879-8709-4bca3d9863c6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.495118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1b66d-f4dd-4c49-9d5d-9d9d4770f681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.496004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1b66d-f4dd-4c49-9d5d-9d9d4770f681 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.496375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1b66d-f4dd-4c49-9d5d-9d9d4770f681 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.496652 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1b66d-f4dd-4c49-9d5d-9d9d4770f681 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.506094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1b66d-f4dd-4c49-9d5d-9d9d4770f681 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:59.507822 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1b66d-f4dd-4c49-9d5d-9d9d4770f681 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:59.519883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0e7bb-e2f9-4925-9754-0cfea4b346d7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-cca1b66d-f4dd-4c49-9d5d-9d9d4770f681 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 09:59:59.524488 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 09:59:59.525559 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 325/666] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 09:59:59.527285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1b66d-f4dd-4c49-9d5d-9d9d4770f681 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:59.563586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658e370-8987-443d-963e-9aedacdb421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.564313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e370-8987-443d-963e-9aedacdb421e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.564613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e370-8987-443d-963e-9aedacdb421e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.564887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658e370-8987-443d-963e-9aedacdb421e None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.565739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658e370-8987-443d-963e-9aedacdb421e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:59.566091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658e370-8987-443d-963e-9aedacdb421e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:59.568144 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658e370-8987-443d-963e-9aedacdb421e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:59.581182 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 342/667] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:59.582047 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658e370-8987-443d-963e-9aedacdb421e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:59.649707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1b66d-f4dd-4c49-9d5d-9d9d4770f681 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.652023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:59.663373 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:59.685358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826a755-ae01-4292-bad0-faf4a4657aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.686023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826a755-ae01-4292-bad0-faf4a4657aca None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.686332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826a755-ae01-4292-bad0-faf4a4657aca None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.686609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826a755-ae01-4292-bad0-faf4a4657aca None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.687334 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826a755-ae01-4292-bad0-faf4a4657aca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:59.687635 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826a755-ae01-4292-bad0-faf4a4657aca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:59.689014 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826a755-ae01-4292-bad0-faf4a4657aca None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:59.700380 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658e370-8987-443d-963e-9aedacdb421e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.701021 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 326/668] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:59.701823 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826a755-ae01-4292-bad0-faf4a4657aca None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:59.739426 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f56e7-42b3-49bf-9d47-fa68ceb4e6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.740357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f56e7-42b3-49bf-9d47-fa68ceb4e6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.740778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f56e7-42b3-49bf-9d47-fa68ceb4e6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.741209 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f56e7-42b3-49bf-9d47-fa68ceb4e6d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.742117 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f56e7-42b3-49bf-9d47-fa68ceb4e6d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:59.742594 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f56e7-42b3-49bf-9d47-fa68ceb4e6d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:59.744136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f56e7-42b3-49bf-9d47-fa68ceb4e6d1 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:59.753128 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 343/669] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:59.754264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f56e7-42b3-49bf-9d47-fa68ceb4e6d1 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:59.818416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826a755-ae01-4292-bad0-faf4a4657aca None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.858453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee144c18-8e9e-4304-b76d-caf2237ac991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.859410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144c18-8e9e-4304-b76d-caf2237ac991 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.859882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144c18-8e9e-4304-b76d-caf2237ac991 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.860144 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144c18-8e9e-4304-b76d-caf2237ac991 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.860994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee144c18-8e9e-4304-b76d-caf2237ac991 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:59.861385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee144c18-8e9e-4304-b76d-caf2237ac991 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:59.862747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee144c18-8e9e-4304-b76d-caf2237ac991 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:59.869543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f56e7-42b3-49bf-9d47-fa68ceb4e6d1 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 09:59:59.886262 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 327/670] 10.222.0.16 () {56 vars in 1139 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:59.886262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee144c18-8e9e-4304-b76d-caf2237ac991 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 09:59:59.891333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:59.911793 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980dbef-79c8-490c-ac7f-914cd36ea660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 09:59:59.912541 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980dbef-79c8-490c-ac7f-914cd36ea660 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 09:59:59.912824 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980dbef-79c8-490c-ac7f-914cd36ea660 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 09:59:59.913101 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980dbef-79c8-490c-ac7f-914cd36ea660 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 09:59:59.913920 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980dbef-79c8-490c-ac7f-914cd36ea660 None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=b97736e49e2a4bfb9cf7093cfa8b2b16) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 09:59:59.914248 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980dbef-79c8-490c-ac7f-914cd36ea660 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 09:59:59.915748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980dbef-79c8-490c-ac7f-914cd36ea660 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 09:59:59.916602 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 09:59:59.922652 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 344/671] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/role_assignments?user.id=b97736e49e2a4bfb9cf7093cfa8b2b16&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 09:59:59.923749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980dbef-79c8-490c-ac7f-914cd36ea660 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:00.011095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee144c18-8e9e-4304-b76d-caf2237ac991 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:00.051531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76abca-b907-450e-9e18-84e4acc331a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:00.052872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76abca-b907-450e-9e18-84e4acc331a6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:00.053362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76abca-b907-450e-9e18-84e4acc331a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:00.053869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76abca-b907-450e-9e18-84e4acc331a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:00.062439 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76abca-b907-450e-9e18-84e4acc331a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:00.063451 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980dbef-79c8-490c-ac7f-914cd36ea660 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:00.064553 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76abca-b907-450e-9e18-84e4acc331a6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-dd76abca-b907-450e-9e18-84e4acc331a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:00.081068 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:00.082108 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:00.083498 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:00.083498 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:00.083498 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:00.083498 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:00.083498 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:00.083498 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:00.083498 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:00.083498 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:00.091133 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 328/672] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:00.092593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76abca-b907-450e-9e18-84e4acc331a6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:00.104956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c2096-8fc7-48c4-82d7-0353e5e70aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:00.106638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c2096-8fc7-48c4-82d7-0353e5e70aab None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:00.107081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c2096-8fc7-48c4-82d7-0353e5e70aab None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:00.107536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c2096-8fc7-48c4-82d7-0353e5e70aab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:00.123974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c2096-8fc7-48c4-82d7-0353e5e70aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:00.126198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c2096-8fc7-48c4-82d7-0353e5e70aab None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-985c2096-8fc7-48c4-82d7-0353e5e70aab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:00.149461 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:00.150533 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:00.151468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:00.151468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:00.151468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:00.151468 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:00.151468 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 345/673] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 09:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:00.152297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c2096-8fc7-48c4-82d7-0353e5e70aab None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:00.192312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:00.202940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:00.221510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76abca-b907-450e-9e18-84e4acc331a6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:00.265291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f39838-3a26-421f-8e38-d3da88988a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:00.266374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f39838-3a26-421f-8e38-d3da88988a30 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:00.266890 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f39838-3a26-421f-8e38-d3da88988a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:00.267360 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f39838-3a26-421f-8e38-d3da88988a30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:00.286683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c2096-8fc7-48c4-82d7-0353e5e70aab None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:00.330837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480a4f5-8d8d-4b75-adef-4468d220bb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:00.331771 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480a4f5-8d8d-4b75-adef-4468d220bb6b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:00.332060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480a4f5-8d8d-4b75-adef-4468d220bb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:00.332336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480a4f5-8d8d-4b75-adef-4468d220bb6b None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:00.333512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480a4f5-8d8d-4b75-adef-4468d220bb6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:00.333820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480a4f5-8d8d-4b75-adef-4468d220bb6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:00.335273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480a4f5-8d8d-4b75-adef-4468d220bb6b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:00.338933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f39838-3a26-421f-8e38-d3da88988a30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:00.340616 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f39838-3a26-421f-8e38-d3da88988a30 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:00.351673 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 346/674] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:00.353725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480a4f5-8d8d-4b75-adef-4468d220bb6b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:00.451398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:00.463848 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 329/675] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 10:00:00.465146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f39838-3a26-421f-8e38-d3da88988a30 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:00.477588 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480a4f5-8d8d-4b75-adef-4468d220bb6b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:00.498718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:00.524115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa29cf-e114-4154-9055-a9814dcc1cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:00.524783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa29cf-e114-4154-9055-a9814dcc1cae None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:00.525061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa29cf-e114-4154-9055-a9814dcc1cae None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:00.525380 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa29cf-e114-4154-9055-a9814dcc1cae None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:00.526428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa29cf-e114-4154-9055-a9814dcc1cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:00.526735 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa29cf-e114-4154-9055-a9814dcc1cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:00.528135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa29cf-e114-4154-9055-a9814dcc1cae None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:00.546340 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 347/676] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:00.546340 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa29cf-e114-4154-9055-a9814dcc1cae None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:00.590791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f39838-3a26-421f-8e38-d3da88988a30 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:00.636131 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0af1d-f2bd-4f10-8a08-71bacfe3657e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:00.636937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0af1d-f2bd-4f10-8a08-71bacfe3657e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:00.637321 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0af1d-f2bd-4f10-8a08-71bacfe3657e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:00.637697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0af1d-f2bd-4f10-8a08-71bacfe3657e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:00.638682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0af1d-f2bd-4f10-8a08-71bacfe3657e None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=dc11037d649241608ea1378cf4692dad) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:00.639147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0af1d-f2bd-4f10-8a08-71bacfe3657e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:00.640795 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0af1d-f2bd-4f10-8a08-71bacfe3657e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:00.647753 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 330/677] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:00 2022] GET /identity/v3/role_assignments?user.id=dc11037d649241608ea1378cf4692dad&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:00.648946 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0af1d-f2bd-4f10-8a08-71bacfe3657e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:00.686485 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa29cf-e114-4154-9055-a9814dcc1cae None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:00.710094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:00.737472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac1e87-aa96-4d7d-914a-8af93d8b5f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:00.739032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac1e87-aa96-4d7d-914a-8af93d8b5f87 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:00.739475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac1e87-aa96-4d7d-914a-8af93d8b5f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:00.740437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac1e87-aa96-4d7d-914a-8af93d8b5f87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:00.760331 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac1e87-aa96-4d7d-914a-8af93d8b5f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:00.761876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac1e87-aa96-4d7d-914a-8af93d8b5f87 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:00.767983 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0af1d-f2bd-4f10-8a08-71bacfe3657e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-fcac1e87-aa96-4d7d-914a-8af93d8b5f87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:00.785906 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:00.787049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:00.788146 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:00.788146 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:00.788146 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:00.788146 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:00.794375 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 348/678] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:00.795854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac1e87-aa96-4d7d-914a-8af93d8b5f87 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:00.806297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:00.807586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c4b6c-7d16-4f7b-a486-1fa114fed9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:00.807847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4b6c-7d16-4f7b-a486-1fa114fed9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:00.808264 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4b6c-7d16-4f7b-a486-1fa114fed9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:00.808690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4b6c-7d16-4f7b-a486-1fa114fed9b5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:00.818762 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c4b6c-7d16-4f7b-a486-1fa114fed9b5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:00.819225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c4b6c-7d16-4f7b-a486-1fa114fed9b5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5c9c4b6c-7d16-4f7b-a486-1fa114fed9b5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:00.847182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:00.848541 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:00.849702 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:00.849702 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:00.849702 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:00.849702 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:00.849702 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:00.849702 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:00.849702 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:00.849702 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:00.849702 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 10:00:00.849702 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:00.852586 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 331/679] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:00.862276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c4b6c-7d16-4f7b-a486-1fa114fed9b5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:00.970658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac1e87-aa96-4d7d-914a-8af93d8b5f87 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:01.012442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde44d60-b2d8-47ca-8868-def1a1402e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:01.013431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde44d60-b2d8-47ca-8868-def1a1402e5c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:01.013853 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde44d60-b2d8-47ca-8868-def1a1402e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:01.014183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c4b6c-7d16-4f7b-a486-1fa114fed9b5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:01.014449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde44d60-b2d8-47ca-8868-def1a1402e5c None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:01.015930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde44d60-b2d8-47ca-8868-def1a1402e5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:01.016406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde44d60-b2d8-47ca-8868-def1a1402e5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:01.017932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde44d60-b2d8-47ca-8868-def1a1402e5c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:01.030457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:01.034355 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 349/680] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:01.036451 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde44d60-b2d8-47ca-8868-def1a1402e5c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:01.055608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d723e-7635-42b6-9dae-37dcd358817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:01.056656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d723e-7635-42b6-9dae-37dcd358817c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:01.057143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d723e-7635-42b6-9dae-37dcd358817c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:01.057709 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d723e-7635-42b6-9dae-37dcd358817c None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:01.058845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d723e-7635-42b6-9dae-37dcd358817c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:01.059450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d723e-7635-42b6-9dae-37dcd358817c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:01.061324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d723e-7635-42b6-9dae-37dcd358817c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:01.081742 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 332/681] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jul 5 10:00:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:01.088227 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d723e-7635-42b6-9dae-37dcd358817c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:01.099399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:01.137755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde44d60-b2d8-47ca-8868-def1a1402e5c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:01.173247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c01804-81c9-4743-9dd9-8691f75f858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:01.174105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c01804-81c9-4743-9dd9-8691f75f858a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:01.174423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c01804-81c9-4743-9dd9-8691f75f858a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:01.174697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c01804-81c9-4743-9dd9-8691f75f858a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:01.182156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c01804-81c9-4743-9dd9-8691f75f858a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:01.183690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c01804-81c9-4743-9dd9-8691f75f858a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-05c01804-81c9-4743-9dd9-8691f75f858a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:01.197746 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:01.198807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:01.199727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:01.199727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:01.199727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:01.199727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:01.199727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:01.199727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:01.199727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:01.199727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:01.204436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d723e-7635-42b6-9dae-37dcd358817c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:01.204983 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 350/682] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:01.206289 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c01804-81c9-4743-9dd9-8691f75f858a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:01.243034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd5926-edf8-4647-a37b-76bfabed8269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:01.244268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd5926-edf8-4647-a37b-76bfabed8269 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:01.244765 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd5926-edf8-4647-a37b-76bfabed8269 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:01.245288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd5926-edf8-4647-a37b-76bfabed8269 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:01.303865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:01.335131 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd5926-edf8-4647-a37b-76bfabed8269 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:01.337024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd5926-edf8-4647-a37b-76bfabed8269 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:01.408527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c01804-81c9-4743-9dd9-8691f75f858a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:01.448456 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 333/683] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 10:00:01.454712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd5926-edf8-4647-a37b-76bfabed8269 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:01.467385 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:01.507622 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed3c6a-bc49-4b98-9060-df4b8bf016a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:01.508380 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed3c6a-bc49-4b98-9060-df4b8bf016a0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:01.508738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed3c6a-bc49-4b98-9060-df4b8bf016a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:01.509056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed3c6a-bc49-4b98-9060-df4b8bf016a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:01.509966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed3c6a-bc49-4b98-9060-df4b8bf016a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:01.510779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed3c6a-bc49-4b98-9060-df4b8bf016a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:01.511934 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed3c6a-bc49-4b98-9060-df4b8bf016a0 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:01.542083 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:01.545255 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 351/684] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:01.551233 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed3c6a-bc49-4b98-9060-df4b8bf016a0 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:01.581624 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd5926-edf8-4647-a37b-76bfabed8269 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:01.626071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d3865-c87b-4594-b9fa-ca3af8f2c931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:01.627047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d3865-c87b-4594-b9fa-ca3af8f2c931 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:01.627482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d3865-c87b-4594-b9fa-ca3af8f2c931 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:01.627769 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d3865-c87b-4594-b9fa-ca3af8f2c931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:01.645448 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d3865-c87b-4594-b9fa-ca3af8f2c931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:01.647071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d3865-c87b-4594-b9fa-ca3af8f2c931 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:01.665593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed3c6a-bc49-4b98-9060-df4b8bf016a0 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1f8d3865-c87b-4594-b9fa-ca3af8f2c931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:01.666639 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:01.667487 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:01.668386 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:01.668386 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:01.668386 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:01.668386 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:01.672532 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 334/685] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:01.673874 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f03a-55be-4f97-8078-c3ee112f50da None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:01.674168 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f03a-55be-4f97-8078-c3ee112f50da None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:01.674491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f03a-55be-4f97-8078-c3ee112f50da None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:01.679894 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 335/686] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:01.679894 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3f03a-55be-4f97-8078-c3ee112f50da None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:01.708086 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a85b53-9831-45a8-abd1-76a486ec913a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:01.708770 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a85b53-9831-45a8-abd1-76a486ec913a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:01.709055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a85b53-9831-45a8-abd1-76a486ec913a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:01.709336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a85b53-9831-45a8-abd1-76a486ec913a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:01.710326 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a85b53-9831-45a8-abd1-76a486ec913a None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=e0865f21da0a4c95a84c0099a1b9a78b) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:01.710649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a85b53-9831-45a8-abd1-76a486ec913a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:01.712210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a85b53-9831-45a8-abd1-76a486ec913a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:01.718817 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 352/687] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:01 2022] GET /identity/v3/role_assignments?user.id=e0865f21da0a4c95a84c0099a1b9a78b&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:01.720033 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b885f4-3592-4390-bccf-bf425f82c7a5 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:01.720322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b885f4-3592-4390-bccf-bf425f82c7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:01.720604 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b885f4-3592-4390-bccf-bf425f82c7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:01.754873 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-93b885f4-3592-4390-bccf-bf425f82c7a5 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:01.756084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-93b885f4-3592-4390-bccf-bf425f82c7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:01.797006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3f03a-55be-4f97-8078-c3ee112f50da None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:01.826131 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:01.838316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea06853-a9ef-426f-9dad-66fc386473f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:01.839601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea06853-a9ef-426f-9dad-66fc386473f3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:01.840081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea06853-a9ef-426f-9dad-66fc386473f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:01.840560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea06853-a9ef-426f-9dad-66fc386473f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:01.841202 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:01.842468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea06853-a9ef-426f-9dad-66fc386473f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:01.843000 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea06853-a9ef-426f-9dad-66fc386473f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:01.844824 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea06853-a9ef-426f-9dad-66fc386473f3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:01.859255 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 336/688] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:01.864828 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea06853-a9ef-426f-9dad-66fc386473f3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:01.906584 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 353/689] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 10:00:01.939382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b885f4-3592-4390-bccf-bf425f82c7a5 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:02.016431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea06853-a9ef-426f-9dad-66fc386473f3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:02.052596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5adf8-1320-479c-8138-291a70c257bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:02.052806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b885f4-3592-4390-bccf-bf425f82c7a5 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:02.054382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5adf8-1320-479c-8138-291a70c257bb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:02.054987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5adf8-1320-479c-8138-291a70c257bb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:02.055451 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5adf8-1320-479c-8138-291a70c257bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:02.062961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5adf8-1320-479c-8138-291a70c257bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:02.064833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5adf8-1320-479c-8138-291a70c257bb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-35b5adf8-1320-479c-8138-291a70c257bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:02.080509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:02.081422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:02.082772 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:02.082772 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:02.082772 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:02.082772 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:02.082772 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:02.082772 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:02.082772 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:02.082772 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:02.085282 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 337/690] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:02.087654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0452-d5c8-4de2-9292-9b804b9eb3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:02.088619 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0452-d5c8-4de2-9292-9b804b9eb3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:02.088908 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0452-d5c8-4de2-9292-9b804b9eb3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:02.089238 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0452-d5c8-4de2-9292-9b804b9eb3e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:02.090732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5adf8-1320-479c-8138-291a70c257bb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:02.101815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c0452-d5c8-4de2-9292-9b804b9eb3e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-143c0452-d5c8-4de2-9292-9b804b9eb3e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:02.109423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:02.110691 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:02.111609 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:02.114339 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 354/691] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:02.116766 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0452-d5c8-4de2-9292-9b804b9eb3e9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:02.154634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:02.158528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:02.204808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5adf8-1320-479c-8138-291a70c257bb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:02.219825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0452-d5c8-4de2-9292-9b804b9eb3e9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:02.241404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292ab12-ad64-411f-abd9-ee049772cebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:02.242449 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292ab12-ad64-411f-abd9-ee049772cebc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:02.242995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292ab12-ad64-411f-abd9-ee049772cebc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:02.243599 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292ab12-ad64-411f-abd9-ee049772cebc None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:02.244591 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292ab12-ad64-411f-abd9-ee049772cebc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:02.245090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292ab12-ad64-411f-abd9-ee049772cebc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:02.246893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292ab12-ad64-411f-abd9-ee049772cebc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:02.255250 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a673bb5-3f99-4ce3-92db-d627bb4af4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:02.255760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a673bb5-3f99-4ce3-92db-d627bb4af4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:02.256042 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a673bb5-3f99-4ce3-92db-d627bb4af4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:02.256314 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a673bb5-3f99-4ce3-92db-d627bb4af4a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:02.257071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a673bb5-3f99-4ce3-92db-d627bb4af4a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:02.257375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a673bb5-3f99-4ce3-92db-d627bb4af4a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:02.259606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a673bb5-3f99-4ce3-92db-d627bb4af4a9 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:02.262735 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 338/692] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:02.268689 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292ab12-ad64-411f-abd9-ee049772cebc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:02.284602 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 355/693] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:00:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:02.293674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a673bb5-3f99-4ce3-92db-d627bb4af4a9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:02.374884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292ab12-ad64-411f-abd9-ee049772cebc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:02.378642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:02.412868 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c198631-ebfb-4b8a-b265-493807b35ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:02.413701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c198631-ebfb-4b8a-b265-493807b35ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:02.414079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c198631-ebfb-4b8a-b265-493807b35ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:02.414472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c198631-ebfb-4b8a-b265-493807b35ff4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:02.415434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c198631-ebfb-4b8a-b265-493807b35ff4 None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=dbfded29b70140a387f7b8c07c296861) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:02.415847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c198631-ebfb-4b8a-b265-493807b35ff4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:02.418039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c198631-ebfb-4b8a-b265-493807b35ff4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:02.425031 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 339/694] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:02 2022] GET /identity/v3/role_assignments?user.id=dbfded29b70140a387f7b8c07c296861&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:02.475507 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71e657-539e-4884-93d2-9b652044201d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:02.475870 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71e657-539e-4884-93d2-9b652044201d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:02.476051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71e657-539e-4884-93d2-9b652044201d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:02.477473 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 340/695] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:02.481901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:02.486072 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a673bb5-3f99-4ce3-92db-d627bb4af4a9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:02.497493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13b095-cdf8-493d-a6f2-c7804df26b0a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:02.497939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13b095-cdf8-493d-a6f2-c7804df26b0a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:02.498245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13b095-cdf8-493d-a6f2-c7804df26b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:02.533904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7c13b095-cdf8-493d-a6f2-c7804df26b0a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:02.535467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7c13b095-cdf8-493d-a6f2-c7804df26b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:02.566833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adbb56-86d4-4741-b673-0400434b68b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:02.567675 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adbb56-86d4-4741-b673-0400434b68b5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:02.568127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adbb56-86d4-4741-b673-0400434b68b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:02.568415 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adbb56-86d4-4741-b673-0400434b68b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:02.590483 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adbb56-86d4-4741-b673-0400434b68b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:02.591991 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adbb56-86d4-4741-b673-0400434b68b5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-e2adbb56-86d4-4741-b673-0400434b68b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:02.625779 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:02.626794 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:02.627689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:02.627689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:02.627689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:02.627689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:02.627689 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 356/696] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:02.633045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adbb56-86d4-4741-b673-0400434b68b5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:02.655582 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 341/697] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 10:00:02.681417 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13b095-cdf8-493d-a6f2-c7804df26b0a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:02.764452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adbb56-86d4-4741-b673-0400434b68b5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:02.796797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:02.802830 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13b095-cdf8-493d-a6f2-c7804df26b0a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:02.806800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4573b37-a2bc-4182-babd-501b72465502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:02.807736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4573b37-a2bc-4182-babd-501b72465502 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:02.808247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4573b37-a2bc-4182-babd-501b72465502 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:02.808749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4573b37-a2bc-4182-babd-501b72465502 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:02.809698 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4573b37-a2bc-4182-babd-501b72465502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:02.810365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4573b37-a2bc-4182-babd-501b72465502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:02.811971 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4573b37-a2bc-4182-babd-501b72465502 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:02.826473 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 357/698] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:02.832101 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4573b37-a2bc-4182-babd-501b72465502 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:02.843162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89b119-c811-47bd-93b3-d3bb84ad7be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:02.844007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89b119-c811-47bd-93b3-d3bb84ad7be0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:02.844294 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89b119-c811-47bd-93b3-d3bb84ad7be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:02.844571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89b119-c811-47bd-93b3-d3bb84ad7be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:02.850789 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:02.859027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89b119-c811-47bd-93b3-d3bb84ad7be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:02.860509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89b119-c811-47bd-93b3-d3bb84ad7be0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-de89b119-c811-47bd-93b3-d3bb84ad7be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:02.883092 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:02.884142 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:02.891175 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:02.891175 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:02.891175 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:02.891175 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:02.891175 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 342/699] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:02.891885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89b119-c811-47bd-93b3-d3bb84ad7be0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:02.960290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4573b37-a2bc-4182-babd-501b72465502 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:03.002559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86a775-7c98-4443-a424-15644fafc75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:03.003517 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86a775-7c98-4443-a424-15644fafc75a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:03.003892 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86a775-7c98-4443-a424-15644fafc75a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:03.004294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86a775-7c98-4443-a424-15644fafc75a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:03.013642 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86a775-7c98-4443-a424-15644fafc75a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:03.015561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86a775-7c98-4443-a424-15644fafc75a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:03.023804 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89b119-c811-47bd-93b3-d3bb84ad7be0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-2c86a775-7c98-4443-a424-15644fafc75a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:03.033509 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:03.034762 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:03.035703 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:03.035703 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:03.035703 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:03.035703 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:03.035703 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:03.035703 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:03.035703 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:03.035703 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:03.035703 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:03.035703 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:03.036236 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 358/700] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jul 5 10:00:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:03.043138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86a775-7c98-4443-a424-15644fafc75a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:03.107356 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d39cf-ff26-4654-ae24-86633e3b3d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:03.108035 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d39cf-ff26-4654-ae24-86633e3b3d39 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:03.108321 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d39cf-ff26-4654-ae24-86633e3b3d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:03.108596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d39cf-ff26-4654-ae24-86633e3b3d39 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:03.109335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d39cf-ff26-4654-ae24-86633e3b3d39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:03.109641 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d39cf-ff26-4654-ae24-86633e3b3d39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:03.111897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d39cf-ff26-4654-ae24-86633e3b3d39 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:03.132547 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 343/701] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:03.138191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d39cf-ff26-4654-ae24-86633e3b3d39 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:03.151358 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:03.160574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:03.216417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86a775-7c98-4443-a424-15644fafc75a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:03.282777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d831a-e45f-4c2b-9bfd-5796e8dbda47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:03.283475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d831a-e45f-4c2b-9bfd-5796e8dbda47 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:03.283755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d831a-e45f-4c2b-9bfd-5796e8dbda47 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:03.284020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d831a-e45f-4c2b-9bfd-5796e8dbda47 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:03.284891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d831a-e45f-4c2b-9bfd-5796e8dbda47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:03.285199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d831a-e45f-4c2b-9bfd-5796e8dbda47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:03.286803 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d831a-e45f-4c2b-9bfd-5796e8dbda47 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:03.295068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d39cf-ff26-4654-ae24-86633e3b3d39 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:03.324830 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 359/702] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jul 5 10:00:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:03.330992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d831a-e45f-4c2b-9bfd-5796e8dbda47 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:03.364094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddeaf9-16a8-4f4c-8475-030087a50585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:03.364788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddeaf9-16a8-4f4c-8475-030087a50585 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:03.365075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddeaf9-16a8-4f4c-8475-030087a50585 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:03.365355 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddeaf9-16a8-4f4c-8475-030087a50585 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:03.366079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddeaf9-16a8-4f4c-8475-030087a50585 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:03.367558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddeaf9-16a8-4f4c-8475-030087a50585 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:03.405105 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 344/703] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 10:00:03 2022] POST /identity/v3/users => generated 254 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:03.414389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77823-4365-4585-8ece-78fd4804aeac None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:03.414591 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77823-4365-4585-8ece-78fd4804aeac None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:03.414853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77823-4365-4585-8ece-78fd4804aeac None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:03.416685 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 345/704] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:03.422782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229aebc-f0d3-4667-948f-dea2fbbf8bae None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:03.423067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229aebc-f0d3-4667-948f-dea2fbbf8bae None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:03.423343 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229aebc-f0d3-4667-948f-dea2fbbf8bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:03.443098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5229aebc-f0d3-4667-948f-dea2fbbf8bae None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:03.444325 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5229aebc-f0d3-4667-948f-dea2fbbf8bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:03.512318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:03.523045 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 346/705] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 10:00:03.546095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229aebc-f0d3-4667-948f-dea2fbbf8bae None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:03.565215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d831a-e45f-4c2b-9bfd-5796e8dbda47 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:03.622489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:03.630667 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fe56e-112d-4227-bff4-ce2e7a5fc04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:03.631393 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fe56e-112d-4227-bff4-ce2e7a5fc04c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:03.631672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fe56e-112d-4227-bff4-ce2e7a5fc04c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:03.631962 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fe56e-112d-4227-bff4-ce2e7a5fc04c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:03.654547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229aebc-f0d3-4667-948f-dea2fbbf8bae None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:03.691999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb23b4-84fd-470e-b2f0-45a0f9fe88d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:03.692798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb23b4-84fd-470e-b2f0-45a0f9fe88d9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:03.693082 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb23b4-84fd-470e-b2f0-45a0f9fe88d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:03.693358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb23b4-84fd-470e-b2f0-45a0f9fe88d9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:03.694016 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb23b4-84fd-470e-b2f0-45a0f9fe88d9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:03.695432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb23b4-84fd-470e-b2f0-45a0f9fe88d9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-eefb23b4-84fd-470e-b2f0-45a0f9fe88d9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:03.711285 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:03.712303 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:03.713581 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 347/706] 10.222.0.16 () {56 vars in 1129 bytes} [Tue Jul 5 10:00:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:03.718556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb23b4-84fd-470e-b2f0-45a0f9fe88d9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:03.732778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574fe56e-112d-4227-bff4-ce2e7a5fc04c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:03.734262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574fe56e-112d-4227-bff4-ce2e7a5fc04c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:03.779237 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:03.824092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb23b4-84fd-470e-b2f0-45a0f9fe88d9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:03.839646 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 360/707] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 10:00:03.845819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fe56e-112d-4227-bff4-ce2e7a5fc04c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:03.858619 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd2f2a-a4e4-46db-b312-5d1bc7bd7e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:03.859299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd2f2a-a4e4-46db-b312-5d1bc7bd7e1a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:03.859578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd2f2a-a4e4-46db-b312-5d1bc7bd7e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:03.859850 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd2f2a-a4e4-46db-b312-5d1bc7bd7e1a None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:03.860565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd2f2a-a4e4-46db-b312-5d1bc7bd7e1a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:03.860866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd2f2a-a4e4-46db-b312-5d1bc7bd7e1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:03.862258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd2f2a-a4e4-46db-b312-5d1bc7bd7e1a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:03.867268 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 348/708] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jul 5 10:00:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:03.872817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd2f2a-a4e4-46db-b312-5d1bc7bd7e1a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:03.878412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:03.955070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fe56e-112d-4227-bff4-ce2e7a5fc04c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:03.974840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd2f2a-a4e4-46db-b312-5d1bc7bd7e1a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:03.992307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efd1d2-b567-4641-b46f-378baf6e60b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:03.993271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efd1d2-b567-4641-b46f-378baf6e60b4 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:03.993548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efd1d2-b567-4641-b46f-378baf6e60b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:03.993818 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efd1d2-b567-4641-b46f-378baf6e60b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:03.999463 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:04.007596 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efd1d2-b567-4641-b46f-378baf6e60b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:04.009357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efd1d2-b567-4641-b46f-378baf6e60b4 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:04.010917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030d6e0-a6a4-4c70-bfe3-b80d79148cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:04.011654 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030d6e0-a6a4-4c70-bfe3-b80d79148cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:04.011940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030d6e0-a6a4-4c70-bfe3-b80d79148cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:04.012264 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030d6e0-a6a4-4c70-bfe3-b80d79148cf1 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:04.012978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030d6e0-a6a4-4c70-bfe3-b80d79148cf1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:04.013284 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030d6e0-a6a4-4c70-bfe3-b80d79148cf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:04.014699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030d6e0-a6a4-4c70-bfe3-b80d79148cf1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:04.021100 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 349/709] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jul 5 10:00:03 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-99efd1d2-b567-4641-b46f-378baf6e60b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:04.028434 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:04.030652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:04.032695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:04.032695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:04.032695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:04.032695 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:04.032695 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 361/710] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:04.039348 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030d6e0-a6a4-4c70-bfe3-b80d79148cf1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:04.185178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030d6e0-a6a4-4c70-bfe3-b80d79148cf1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:04.222914 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5794e77-a023-4f91-a6b5-8c50a134b5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:04.223933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5794e77-a023-4f91-a6b5-8c50a134b5ae None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:04.224217 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5794e77-a023-4f91-a6b5-8c50a134b5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:04.224561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5794e77-a023-4f91-a6b5-8c50a134b5ae None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:04.225292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5794e77-a023-4f91-a6b5-8c50a134b5ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:04.225597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5794e77-a023-4f91-a6b5-8c50a134b5ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:04.227102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5794e77-a023-4f91-a6b5-8c50a134b5ae None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:04.243948 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 350/711] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:04.245357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efd1d2-b567-4641-b46f-378baf6e60b4 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:04.374008 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:04.394201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efd1d2-b567-4641-b46f-378baf6e60b4 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:04.432472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ebfde-421b-422e-88ef-538b86515fbb None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:04.433093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ebfde-421b-422e-88ef-538b86515fbb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:04.433559 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ebfde-421b-422e-88ef-538b86515fbb None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:04.434454 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e72bc-eace-49f3-b49b-a2a903634a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:04.435265 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e72bc-eace-49f3-b49b-a2a903634a4b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:04.435548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e72bc-eace-49f3-b49b-a2a903634a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:04.435822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e72bc-eace-49f3-b49b-a2a903634a4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:04.440119 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 351/712] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:04.441961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dc478-3870-4043-9be3-3eaf3fd04c5c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:04.442515 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dc478-3870-4043-9be3-3eaf3fd04c5c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:04.443004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dc478-3870-4043-9be3-3eaf3fd04c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:04.451052 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e72bc-eace-49f3-b49b-a2a903634a4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:04.452536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e72bc-eace-49f3-b49b-a2a903634a4b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:04.467203 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-869dc478-3870-4043-9be3-3eaf3fd04c5c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:04.468798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-869dc478-3870-4043-9be3-3eaf3fd04c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-262e72bc-eace-49f3-b49b-a2a903634a4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:04.474465 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:04.475442 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:04.476335 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:04.476335 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:04.476335 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:04.476335 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:04.476335 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:04.476335 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:04.476335 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:04.476335 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:04.485951 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 362/713] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:04.492387 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e72bc-eace-49f3-b49b-a2a903634a4b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:04.556450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:04.591921 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 352/714] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 10:00:04.625772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869dc478-3870-4043-9be3-3eaf3fd04c5c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:04.647438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e72bc-eace-49f3-b49b-a2a903634a4b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:04.717399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb80c7-207f-471f-8f91-7ec20d9e2737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:04.718092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb80c7-207f-471f-8f91-7ec20d9e2737 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:04.718395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb80c7-207f-471f-8f91-7ec20d9e2737 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:04.718672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb80c7-207f-471f-8f91-7ec20d9e2737 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:04.719462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb80c7-207f-471f-8f91-7ec20d9e2737 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:04.719767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb80c7-207f-471f-8f91-7ec20d9e2737 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:04.721221 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb80c7-207f-471f-8f91-7ec20d9e2737 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:04.765699 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 363/715] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:04.766728 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb80c7-207f-471f-8f91-7ec20d9e2737 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:04.771820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:04.834242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869dc478-3870-4043-9be3-3eaf3fd04c5c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:04.883231 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb80c7-207f-471f-8f91-7ec20d9e2737 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:04.918381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0afec-16a4-4928-ab12-5a12af68c481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:04.919045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0afec-16a4-4928-ab12-5a12af68c481 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:04.919329 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0afec-16a4-4928-ab12-5a12af68c481 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:04.919606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0afec-16a4-4928-ab12-5a12af68c481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:04.920438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0afec-16a4-4928-ab12-5a12af68c481 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=fee6ee8f99c34aa281bf02ecd3f23216) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:04.920749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0afec-16a4-4928-ab12-5a12af68c481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:04.922310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0afec-16a4-4928-ab12-5a12af68c481 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:04.928374 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 364/716] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:04 2022] GET /identity/v3/role_assignments?user.id=fee6ee8f99c34aa281bf02ecd3f23216&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=c184a177010144429f495f3290d28221 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:04.930950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465e1e9-260b-4b93-a1c7-19943daedf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:04.931794 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465e1e9-260b-4b93-a1c7-19943daedf94 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:04.932081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465e1e9-260b-4b93-a1c7-19943daedf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:04.932359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465e1e9-260b-4b93-a1c7-19943daedf94 None admin] PATH_INFO: `/v3/services/network` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:04.933039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465e1e9-260b-4b93-a1c7-19943daedf94 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:04.934483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465e1e9-260b-4b93-a1c7-19943daedf94 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5465e1e9-260b-4b93-a1c7-19943daedf94 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:04.968934 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:04.970012 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:04.970896 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 353/717] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:04 2022] GET /identity/v3/services/network => generated 88 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:04.978698 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0afec-16a4-4928-ab12-5a12af68c481 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:05.083107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:05.125874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0afec-16a4-4928-ab12-5a12af68c481 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:05.164023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd576aa-4594-4ca6-bb8e-336e65a3a46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:05.164730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd576aa-4594-4ca6-bb8e-336e65a3a46c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:05.165008 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd576aa-4594-4ca6-bb8e-336e65a3a46c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:05.165280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd576aa-4594-4ca6-bb8e-336e65a3a46c None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:05.166014 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd576aa-4594-4ca6-bb8e-336e65a3a46c None admin] RBAC: Adding query filter params (name=network) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:05.166345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd576aa-4594-4ca6-bb8e-336e65a3a46c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:05.167768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd576aa-4594-4ca6-bb8e-336e65a3a46c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:05.173451 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 365/718] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:05.183531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465e1e9-260b-4b93-a1c7-19943daedf94 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:05.196290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe1a9e-2276-4318-af7c-2391c3492c86 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:05.196769 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe1a9e-2276-4318-af7c-2391c3492c86 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:05.196883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe1a9e-2276-4318-af7c-2391c3492c86 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:05.198473 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 366/719] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:05.211648 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de42f-87e2-4444-82de-62b87bfb716f None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:05.211932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de42f-87e2-4444-82de-62b87bfb716f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:05.212207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de42f-87e2-4444-82de-62b87bfb716f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:05.237365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b93de42f-87e2-4444-82de-62b87bfb716f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:05.239332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b93de42f-87e2-4444-82de-62b87bfb716f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:05.254611 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:05.322876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465e1e9-260b-4b93-a1c7-19943daedf94 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:05.335686 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 367/720] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 10:00:05.361896 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b29ce-d384-4ca8-b495-453dde8f2cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:05.362631 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b29ce-d384-4ca8-b495-453dde8f2cce None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:05.362915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b29ce-d384-4ca8-b495-453dde8f2cce None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:05.363189 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b29ce-d384-4ca8-b495-453dde8f2cce None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:05.364020 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b29ce-d384-4ca8-b495-453dde8f2cce None admin] RBAC: Adding query filter params (type=network) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:05.364325 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b29ce-d384-4ca8-b495-453dde8f2cce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:05.365736 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b29ce-d384-4ca8-b495-453dde8f2cce None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:05.372079 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93de42f-87e2-4444-82de-62b87bfb716f None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:05.383816 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 354/721] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:05.435903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:05.479448 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93de42f-87e2-4444-82de-62b87bfb716f None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:05.515521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425fce6-c59d-44be-afd0-362c20a3da16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:05.516348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425fce6-c59d-44be-afd0-362c20a3da16 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:05.516624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425fce6-c59d-44be-afd0-362c20a3da16 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:05.516940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425fce6-c59d-44be-afd0-362c20a3da16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:05.528678 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425fce6-c59d-44be-afd0-362c20a3da16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-6425fce6-c59d-44be-afd0-362c20a3da16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:05.536547 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:05.537580 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:05.538493 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 368/722] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:05.545966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b29ce-d384-4ca8-b495-453dde8f2cce None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:05.656920 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b29ce-d384-4ca8-b495-453dde8f2cce None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:05.664813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:05.699189 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12d21e-27f7-4f39-9303-dde515894fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:05.699872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12d21e-27f7-4f39-9303-dde515894fde None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:05.700155 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12d21e-27f7-4f39-9303-dde515894fde None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:05.700429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12d21e-27f7-4f39-9303-dde515894fde None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:05.701190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12d21e-27f7-4f39-9303-dde515894fde None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:05.701495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12d21e-27f7-4f39-9303-dde515894fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:05.702925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12d21e-27f7-4f39-9303-dde515894fde None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:05.712244 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 355/723] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:00:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:05.717648 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425fce6-c59d-44be-afd0-362c20a3da16 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:05.824013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425fce6-c59d-44be-afd0-362c20a3da16 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:05.859396 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:05.880686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edb15e-7f9e-4959-9917-f246fc25ac83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:05.881502 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edb15e-7f9e-4959-9917-f246fc25ac83 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:05.881781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edb15e-7f9e-4959-9917-f246fc25ac83 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:05.882051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edb15e-7f9e-4959-9917-f246fc25ac83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:05.898539 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edb15e-7f9e-4959-9917-f246fc25ac83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:05.899997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edb15e-7f9e-4959-9917-f246fc25ac83 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-f1edb15e-7f9e-4959-9917-f246fc25ac83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:05.922234 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:05.923321 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:05.924279 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:05.924279 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:05.924279 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:05.924279 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:05.924279 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 369/724] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:05.929386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12d21e-27f7-4f39-9303-dde515894fde None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:06.045265 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12d21e-27f7-4f39-9303-dde515894fde None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:06.079279 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d03dbf-0d7f-464c-8708-594ea47e651c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:06.079990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d03dbf-0d7f-464c-8708-594ea47e651c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:06.080274 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d03dbf-0d7f-464c-8708-594ea47e651c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:06.080550 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d03dbf-0d7f-464c-8708-594ea47e651c None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:06.081277 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d03dbf-0d7f-464c-8708-594ea47e651c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:06.081581 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d03dbf-0d7f-464c-8708-594ea47e651c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:06.083356 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d03dbf-0d7f-464c-8708-594ea47e651c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:06.095999 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 356/725] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:06.103346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edb15e-7f9e-4959-9917-f246fc25ac83 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:06.218403 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36431de-1025-4ae2-8a6a-a49134c52f95 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:06.218727 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36431de-1025-4ae2-8a6a-a49134c52f95 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:06.218931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36431de-1025-4ae2-8a6a-a49134c52f95 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:06.220539 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 357/726] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:06.225397 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca4a1-5516-466c-8ec9-b8ead31a81b5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:06.225682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca4a1-5516-466c-8ec9-b8ead31a81b5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:06.225959 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ca4a1-5516-466c-8ec9-b8ead31a81b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:06.241166 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edb15e-7f9e-4959-9917-f246fc25ac83 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:06.244754 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-323ca4a1-5516-466c-8ec9-b8ead31a81b5 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:06.246166 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-323ca4a1-5516-466c-8ec9-b8ead31a81b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:06.273496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:06.305554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bfd3b-f75d-48b3-bb02-6dbf8e01f2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:06.306432 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bfd3b-f75d-48b3-bb02-6dbf8e01f2fc None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:06.306712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bfd3b-f75d-48b3-bb02-6dbf8e01f2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:06.306983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bfd3b-f75d-48b3-bb02-6dbf8e01f2fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:06.319208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bfd3b-f75d-48b3-bb02-6dbf8e01f2fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:06.321021 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bfd3b-f75d-48b3-bb02-6dbf8e01f2fc None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-e54bfd3b-f75d-48b3-bb02-6dbf8e01f2fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:06.344555 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:06.345951 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:06.346970 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:06.346970 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:06.346970 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:06.346970 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:06.346970 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:06.346970 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:06.346970 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:06.346970 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:06.346970 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 10:00:06.346970 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:06.352267 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 370/727] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:06.354730 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 358/728] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 10:00:06.359164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bfd3b-f75d-48b3-bb02-6dbf8e01f2fc None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:06.401292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:06.404379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ca4a1-5516-466c-8ec9-b8ead31a81b5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:06.485213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bfd3b-f75d-48b3-bb02-6dbf8e01f2fc None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:06.535308 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ca4a1-5516-466c-8ec9-b8ead31a81b5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:06.535568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bb2e9-e8cf-41de-b28c-3679df111595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:06.536252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bb2e9-e8cf-41de-b28c-3679df111595 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:06.536598 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bb2e9-e8cf-41de-b28c-3679df111595 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:06.536870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bb2e9-e8cf-41de-b28c-3679df111595 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:06.537818 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bb2e9-e8cf-41de-b28c-3679df111595 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:06.538128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bb2e9-e8cf-41de-b28c-3679df111595 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:06.539609 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bb2e9-e8cf-41de-b28c-3679df111595 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:06.572044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486e131-f904-457c-bbaf-0982f70d5b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:06.572899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486e131-f904-457c-bbaf-0982f70d5b06 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:06.573183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486e131-f904-457c-bbaf-0982f70d5b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:06.573453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486e131-f904-457c-bbaf-0982f70d5b06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:06.580947 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 371/729] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jul 5 10:00:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:06.582063 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bb2e9-e8cf-41de-b28c-3679df111595 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:06.584435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486e131-f904-457c-bbaf-0982f70d5b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-3486e131-f904-457c-bbaf-0982f70d5b06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:06.592414 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:06.593376 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:06.594293 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 359/730] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 10:00:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:06.599574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486e131-f904-457c-bbaf-0982f70d5b06 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:06.620103 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:06.705152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486e131-f904-457c-bbaf-0982f70d5b06 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:06.728645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:06.740568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bb2e9-e8cf-41de-b28c-3679df111595 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:06.742240 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfdc69-df1b-4d00-b10a-cc1bda0a3a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:06.742928 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfdc69-df1b-4d00-b10a-cc1bda0a3a75 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:06.743212 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfdc69-df1b-4d00-b10a-cc1bda0a3a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:06.743489 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfdc69-df1b-4d00-b10a-cc1bda0a3a75 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:06.744242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfdc69-df1b-4d00-b10a-cc1bda0a3a75 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:06.744562 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfdc69-df1b-4d00-b10a-cc1bda0a3a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:06.745966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfdc69-df1b-4d00-b10a-cc1bda0a3a75 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:06.755624 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 360/731] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 10:00:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:06.758680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b11b-e335-48ef-9ff8-84f7bb20eab5 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:06.759995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b11b-e335-48ef-9ff8-84f7bb20eab5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:06.761223 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b11b-e335-48ef-9ff8-84f7bb20eab5 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:06.764010 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 361/732] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:06.768945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910b11b-e335-48ef-9ff8-84f7bb20eab5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:06.787960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05289b-a238-4457-b063-574e6d9e5313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:06.789012 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05289b-a238-4457-b063-574e6d9e5313 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:06.789578 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05289b-a238-4457-b063-574e6d9e5313 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:06.790027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05289b-a238-4457-b063-574e6d9e5313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:06.859123 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:06.860501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05289b-a238-4457-b063-574e6d9e5313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:06.862273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05289b-a238-4457-b063-574e6d9e5313 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:06.880320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910b11b-e335-48ef-9ff8-84f7bb20eab5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:06.918655 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4150ad7-8b2e-4957-b2a0-661f693707f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:06.919456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4150ad7-8b2e-4957-b2a0-661f693707f8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:06.919738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4150ad7-8b2e-4957-b2a0-661f693707f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:06.920011 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4150ad7-8b2e-4957-b2a0-661f693707f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:06.931153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4150ad7-8b2e-4957-b2a0-661f693707f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:06.932725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4150ad7-8b2e-4957-b2a0-661f693707f8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-b4150ad7-8b2e-4957-b2a0-661f693707f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:06.952787 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:06.953912 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:06.954958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:06.954958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:06.954958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:06.954958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:06.954958 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 362/733] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:06.955238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831cdcc-2845-4d2a-81b4-6056d3ac9d76 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:06.955608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831cdcc-2845-4d2a-81b4-6056d3ac9d76 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:06.955749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831cdcc-2845-4d2a-81b4-6056d3ac9d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:06.961929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:06.973421 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 372/734] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 10:00:06.974725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05289b-a238-4457-b063-574e6d9e5313 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:06.981626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1831cdcc-2845-4d2a-81b4-6056d3ac9d76 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:06.982865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1831cdcc-2845-4d2a-81b4-6056d3ac9d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:07.053291 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 363/735] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 10:00:07.054283 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831cdcc-2845-4d2a-81b4-6056d3ac9d76 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:07.079237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05289b-a238-4457-b063-574e6d9e5313 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:07.115229 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22bf09-a0c4-4da2-8de4-7f80618de6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:07.116518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:07.116812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22bf09-a0c4-4da2-8de4-7f80618de6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:07.117999 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22bf09-a0c4-4da2-8de4-7f80618de6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:07.118497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22bf09-a0c4-4da2-8de4-7f80618de6c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:07.119479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22bf09-a0c4-4da2-8de4-7f80618de6c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:07.119974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22bf09-a0c4-4da2-8de4-7f80618de6c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:07.122513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22bf09-a0c4-4da2-8de4-7f80618de6c7 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:07.138411 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 373/736] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:07.139568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22bf09-a0c4-4da2-8de4-7f80618de6c7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:07.156341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831cdcc-2845-4d2a-81b4-6056d3ac9d76 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:07.191916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f392e-eca6-44ac-bfe7-fac57fb9212c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:07.192754 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f392e-eca6-44ac-bfe7-fac57fb9212c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:07.193256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:07.193370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f392e-eca6-44ac-bfe7-fac57fb9212c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:07.194301 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f392e-eca6-44ac-bfe7-fac57fb9212c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:07.206603 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f392e-eca6-44ac-bfe7-fac57fb9212c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:07.208433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f392e-eca6-44ac-bfe7-fac57fb9212c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ee3f392e-eca6-44ac-bfe7-fac57fb9212c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:07.226877 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:07.227809 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:07.229281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:07.229281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:07.229281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:07.229281 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:07.229281 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 364/737] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:07.229281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f392e-eca6-44ac-bfe7-fac57fb9212c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:07.250085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22bf09-a0c4-4da2-8de4-7f80618de6c7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:07.285846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f843b66-814d-4beb-9f1e-efc3aadddd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:07.286911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f843b66-814d-4beb-9f1e-efc3aadddd8b None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:07.287410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f843b66-814d-4beb-9f1e-efc3aadddd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:07.287921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f843b66-814d-4beb-9f1e-efc3aadddd8b None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:07.288937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f843b66-814d-4beb-9f1e-efc3aadddd8b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:07.290630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f843b66-814d-4beb-9f1e-efc3aadddd8b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:07.310274 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 374/738] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 10:00:07 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:07.311528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f843b66-814d-4beb-9f1e-efc3aadddd8b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:07.346250 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f392e-eca6-44ac-bfe7-fac57fb9212c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:07.365718 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:07.407127 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143bf34-fe7e-40fd-a4e8-bb21dfce0020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:07.407975 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143bf34-fe7e-40fd-a4e8-bb21dfce0020 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:07.408322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143bf34-fe7e-40fd-a4e8-bb21dfce0020 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:07.408601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143bf34-fe7e-40fd-a4e8-bb21dfce0020 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:07.420966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143bf34-fe7e-40fd-a4e8-bb21dfce0020 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:07.422732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143bf34-fe7e-40fd-a4e8-bb21dfce0020 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:07.434941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f843b66-814d-4beb-9f1e-efc3aadddd8b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-3143bf34-fe7e-40fd-a4e8-bb21dfce0020 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:07.443021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:07.444030 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:07.449476 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:07.449476 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:07.449476 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:07.449476 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:07.449476 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:07.449476 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:07.449476 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:07.449476 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:07.449476 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:07.449476 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:07.449476 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 365/739] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jul 5 10:00:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:07.450739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:07.452496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143bf34-fe7e-40fd-a4e8-bb21dfce0020 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:07.486874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bc9b7-a77e-430b-8967-6f48b7a54904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:07.487565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bc9b7-a77e-430b-8967-6f48b7a54904 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:07.487846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bc9b7-a77e-430b-8967-6f48b7a54904 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:07.488122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bc9b7-a77e-430b-8967-6f48b7a54904 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:07.489151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bc9b7-a77e-430b-8967-6f48b7a54904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:07.489461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bc9b7-a77e-430b-8967-6f48b7a54904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:07.491240 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bc9b7-a77e-430b-8967-6f48b7a54904 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:07.507198 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 375/740] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:07.514253 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bc9b7-a77e-430b-8967-6f48b7a54904 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:07.594342 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143bf34-fe7e-40fd-a4e8-bb21dfce0020 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:07.638035 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4cac4-9191-46a0-a491-a62a3774ffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:07.638483 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bc9b7-a77e-430b-8967-6f48b7a54904 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:07.638792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4cac4-9191-46a0-a491-a62a3774ffad None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:07.639080 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4cac4-9191-46a0-a491-a62a3774ffad None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:07.639358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4cac4-9191-46a0-a491-a62a3774ffad None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:07.640368 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4cac4-9191-46a0-a491-a62a3774ffad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:07.640697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4cac4-9191-46a0-a491-a62a3774ffad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:07.643014 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4cac4-9191-46a0-a491-a62a3774ffad None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:07.665026 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 366/741] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jul 5 10:00:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:07.666852 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a41af9-9099-487b-85b6-dc202ef8b896 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:07.667437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a41af9-9099-487b-85b6-dc202ef8b896 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:07.667787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a41af9-9099-487b-85b6-dc202ef8b896 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:07.669732 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 367/742] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:07.670778 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a41af9-9099-487b-85b6-dc202ef8b896 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:07.679957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47967c-dcab-4725-bcd9-0cd3f923bf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:07.681196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47967c-dcab-4725-bcd9-0cd3f923bf6f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:07.681661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47967c-dcab-4725-bcd9-0cd3f923bf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:07.682119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47967c-dcab-4725-bcd9-0cd3f923bf6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:07.691227 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47967c-dcab-4725-bcd9-0cd3f923bf6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:07.692997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47967c-dcab-4725-bcd9-0cd3f923bf6f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:07.699682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-9e47967c-dcab-4725-bcd9-0cd3f923bf6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:07.710181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:07.711126 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:07.712029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:07.712029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:07.712029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:07.712029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:07.712029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:07.712029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:07.712029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:07.712029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:07.713825 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 376/743] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:07.715212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3f0f-d23f-4bd5-bc8b-c33360dbbd1b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:07.715642 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3f0f-d23f-4bd5-bc8b-c33360dbbd1b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:07.716080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3f0f-d23f-4bd5-bc8b-c33360dbbd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:07.734734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-588f3f0f-d23f-4bd5-bc8b-c33360dbbd1b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:07.735941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-588f3f0f-d23f-4bd5-bc8b-c33360dbbd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:07.767523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:07.803757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a41af9-9099-487b-85b6-dc202ef8b896 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:07.828245 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 377/744] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 10:00:07.828245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f3f0f-d23f-4bd5-bc8b-c33360dbbd1b None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:07.856095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac331f47-d1ca-4780-a72d-44af807be705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:07.856684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac331f47-d1ca-4780-a72d-44af807be705 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:07.856972 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac331f47-d1ca-4780-a72d-44af807be705 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:07.857249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac331f47-d1ca-4780-a72d-44af807be705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:07.929828 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac331f47-d1ca-4780-a72d-44af807be705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:07.931313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac331f47-d1ca-4780-a72d-44af807be705 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:07.943285 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f3f0f-d23f-4bd5-bc8b-c33360dbbd1b None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:07.958381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:07.981664 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7205e5b-a183-4c20-87cc-16f43cbf0df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:07.982372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7205e5b-a183-4c20-87cc-16f43cbf0df9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:07.982658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7205e5b-a183-4c20-87cc-16f43cbf0df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:07.982930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7205e5b-a183-4c20-87cc-16f43cbf0df9 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:07.983729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7205e5b-a183-4c20-87cc-16f43cbf0df9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:07.984038 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7205e5b-a183-4c20-87cc-16f43cbf0df9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:07.985497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7205e5b-a183-4c20-87cc-16f43cbf0df9 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:07.997676 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 378/745] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:07.998703 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7205e5b-a183-4c20-87cc-16f43cbf0df9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:08.021010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:08.042268 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 368/746] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:07 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 10:00:08.042708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fd3ab-889a-4333-bfe5-2458bbc620c2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:08.042994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fd3ab-889a-4333-bfe5-2458bbc620c2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:08.043271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fd3ab-889a-4333-bfe5-2458bbc620c2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:08.044664 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 369/747] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:08.045200 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fd3ab-889a-4333-bfe5-2458bbc620c2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:08.109066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7205e5b-a183-4c20-87cc-16f43cbf0df9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:08.144260 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855133ef-6ff4-42f6-932c-06e26c7750ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:08.145411 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855133ef-6ff4-42f6-932c-06e26c7750ef None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:08.145966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855133ef-6ff4-42f6-932c-06e26c7750ef None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:08.146271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855133ef-6ff4-42f6-932c-06e26c7750ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:08.151611 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fd3ab-889a-4333-bfe5-2458bbc620c2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:08.157621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855133ef-6ff4-42f6-932c-06e26c7750ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-855133ef-6ff4-42f6-932c-06e26c7750ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:08.165281 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:08.166320 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:08.167181 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 379/748] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:08.167917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855133ef-6ff4-42f6-932c-06e26c7750ef None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:08.182879 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:08.191323 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853294ae-6475-406c-9c5f-18a5501f6c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:08.192032 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853294ae-6475-406c-9c5f-18a5501f6c84 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:08.192313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853294ae-6475-406c-9c5f-18a5501f6c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:08.192584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853294ae-6475-406c-9c5f-18a5501f6c84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:08.193520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853294ae-6475-406c-9c5f-18a5501f6c84 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=dbfded29b70140a387f7b8c07c296861) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:08.193833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853294ae-6475-406c-9c5f-18a5501f6c84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:08.195325 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853294ae-6475-406c-9c5f-18a5501f6c84 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:08.200963 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 370/749] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:08 2022] GET /identity/v3/role_assignments?user.id=dbfded29b70140a387f7b8c07c296861&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=c184a177010144429f495f3290d28221 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:08.202242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91452b-bd13-4864-8674-ca550969b7b3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:08.202547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91452b-bd13-4864-8674-ca550969b7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:08.202820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91452b-bd13-4864-8674-ca550969b7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:08.220836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3a91452b-bd13-4864-8674-ca550969b7b3 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:08.222070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3a91452b-bd13-4864-8674-ca550969b7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:08.279253 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:08.305698 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 371/750] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 10:00:08.307116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855133ef-6ff4-42f6-932c-06e26c7750ef None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:08.311369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91452b-bd13-4864-8674-ca550969b7b3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:08.362413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db851ea0-3e3c-49a7-b107-06847578fe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:08.363647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db851ea0-3e3c-49a7-b107-06847578fe14 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:08.364137 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db851ea0-3e3c-49a7-b107-06847578fe14 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:08.364587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db851ea0-3e3c-49a7-b107-06847578fe14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:08.377323 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db851ea0-3e3c-49a7-b107-06847578fe14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:08.381087 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db851ea0-3e3c-49a7-b107-06847578fe14 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-db851ea0-3e3c-49a7-b107-06847578fe14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:08.398052 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:08.399182 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:08.400100 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:08.400100 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:08.400100 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:08.400100 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:08.402329 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 380/751] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:08.403377 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db851ea0-3e3c-49a7-b107-06847578fe14 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:08.458973 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91452b-bd13-4864-8674-ca550969b7b3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:08.502287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc0414-1e23-4c8c-b204-2f9235f7c415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:08.503572 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc0414-1e23-4c8c-b204-2f9235f7c415 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:08.504423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc0414-1e23-4c8c-b204-2f9235f7c415 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:08.504778 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc0414-1e23-4c8c-b204-2f9235f7c415 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:08.505909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc0414-1e23-4c8c-b204-2f9235f7c415 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:08.506311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc0414-1e23-4c8c-b204-2f9235f7c415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:08.508419 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc0414-1e23-4c8c-b204-2f9235f7c415 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:08.515123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db851ea0-3e3c-49a7-b107-06847578fe14 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:08.520250 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 372/752] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:00:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:08.524587 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:08.526642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc0414-1e23-4c8c-b204-2f9235f7c415 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:08.551548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:08.562505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16ecad-f300-421b-b0db-ee863d9e84b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:08.563506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16ecad-f300-421b-b0db-ee863d9e84b5 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:08.564011 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16ecad-f300-421b-b0db-ee863d9e84b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:08.564545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16ecad-f300-421b-b0db-ee863d9e84b5 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:08.565525 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16ecad-f300-421b-b0db-ee863d9e84b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:08.567846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16ecad-f300-421b-b0db-ee863d9e84b5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:08.587462 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 381/753] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 10:00:08 2022] POST /identity/v3/services => generated 240 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:08.588419 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16ecad-f300-421b-b0db-ee863d9e84b5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:08.661116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc0414-1e23-4c8c-b204-2f9235f7c415 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:08.705710 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f75d0d-a09d-46df-add8-5a5da6599275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:08.706444 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f75d0d-a09d-46df-add8-5a5da6599275 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:08.706741 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f75d0d-a09d-46df-add8-5a5da6599275 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:08.707597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f75d0d-a09d-46df-add8-5a5da6599275 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:08.708460 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f75d0d-a09d-46df-add8-5a5da6599275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:08.708782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f75d0d-a09d-46df-add8-5a5da6599275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:08.710306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f75d0d-a09d-46df-add8-5a5da6599275 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:08.717666 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16ecad-f300-421b-b0db-ee863d9e84b5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:08.724480 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 373/754] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:08.731486 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f75d0d-a09d-46df-add8-5a5da6599275 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:08.764541 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b89220-09a9-4b89-9052-477cc57b804f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:08.765456 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b89220-09a9-4b89-9052-477cc57b804f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:08.765738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b89220-09a9-4b89-9052-477cc57b804f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:08.766016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b89220-09a9-4b89-9052-477cc57b804f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:08.778036 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b89220-09a9-4b89-9052-477cc57b804f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:08.779559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b89220-09a9-4b89-9052-477cc57b804f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-a9b89220-09a9-4b89-9052-477cc57b804f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:08.797627 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:08.798686 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:08.799577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:08.799577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:08.799577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:08.799577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:08.799577 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 382/755] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:08.805013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b89220-09a9-4b89-9052-477cc57b804f None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:08.850454 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:08.881319 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:08.938015 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f75d0d-a09d-46df-add8-5a5da6599275 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:08.949935 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b89220-09a9-4b89-9052-477cc57b804f None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:08.976162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ec1ae-9403-44e3-8113-08cda831370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:08.977071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ec1ae-9403-44e3-8113-08cda831370d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:08.977357 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ec1ae-9403-44e3-8113-08cda831370d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:08.977630 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ec1ae-9403-44e3-8113-08cda831370d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:08.984461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e214d3-b9bc-4e9a-992a-98921d3b22ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:08.984924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ec1ae-9403-44e3-8113-08cda831370d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:08.985141 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e214d3-b9bc-4e9a-992a-98921d3b22ff None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:08.985424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e214d3-b9bc-4e9a-992a-98921d3b22ff None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:08.985696 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e214d3-b9bc-4e9a-992a-98921d3b22ff None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:08.986415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ec1ae-9403-44e3-8113-08cda831370d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:08.986536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e214d3-b9bc-4e9a-992a-98921d3b22ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:08.986741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e214d3-b9bc-4e9a-992a-98921d3b22ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:08.988382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e214d3-b9bc-4e9a-992a-98921d3b22ff None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:09.001110 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 383/756] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-bf7ec1ae-9403-44e3-8113-08cda831370d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:09.002197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:09.003149 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:09.004112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:09.004112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:09.004112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:09.004112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:09.004112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:09.004112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:09.004112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:09.004112 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:09.004112 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 374/757] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:09.007645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e214d3-b9bc-4e9a-992a-98921d3b22ff None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:09.015344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ec1ae-9403-44e3-8113-08cda831370d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:09.158597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ec1ae-9403-44e3-8113-08cda831370d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:09.162525 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:09.196030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743220a-3100-42d5-8a29-a2169457331b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:09.196630 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743220a-3100-42d5-8a29-a2169457331b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:09.196914 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743220a-3100-42d5-8a29-a2169457331b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:09.197190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743220a-3100-42d5-8a29-a2169457331b None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:09.197986 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743220a-3100-42d5-8a29-a2169457331b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:09.198333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743220a-3100-42d5-8a29-a2169457331b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:09.199791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743220a-3100-42d5-8a29-a2169457331b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:09.212897 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 375/758] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:09.216388 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e214d3-b9bc-4e9a-992a-98921d3b22ff None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:09.219047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743220a-3100-42d5-8a29-a2169457331b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:09.224380 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:09.312675 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdefaa-fb3a-4025-84e5-6ae77bb82799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:09.313548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdefaa-fb3a-4025-84e5-6ae77bb82799 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:09.313837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdefaa-fb3a-4025-84e5-6ae77bb82799 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:09.314114 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdefaa-fb3a-4025-84e5-6ae77bb82799 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:09.335125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdefaa-fb3a-4025-84e5-6ae77bb82799 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:09.336612 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdefaa-fb3a-4025-84e5-6ae77bb82799 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:09.346794 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743220a-3100-42d5-8a29-a2169457331b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-dbfdefaa-fb3a-4025-84e5-6ae77bb82799 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:09.367797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:09.368808 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:09.369734 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:09.369734 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:09.369734 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:09.369734 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:09.369734 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:09.369734 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:09.369734 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:09.369734 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:09.369734 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:09.369734 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:09.369734 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 384/759] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jul 5 10:00:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:09.377638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdefaa-fb3a-4025-84e5-6ae77bb82799 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:09.428453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb7999-a3fc-439f-8893-e2305dbde8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:09.429555 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb7999-a3fc-439f-8893-e2305dbde8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:09.430059 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb7999-a3fc-439f-8893-e2305dbde8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:09.430602 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb7999-a3fc-439f-8893-e2305dbde8f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:09.431757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb7999-a3fc-439f-8893-e2305dbde8f3 None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=84a8d62afe964cb49acc236850adbbf3) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:09.432316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb7999-a3fc-439f-8893-e2305dbde8f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:09.433955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb7999-a3fc-439f-8893-e2305dbde8f3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:09.447226 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 376/760] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:09 2022] GET /identity/v3/role_assignments?user.id=84a8d62afe964cb49acc236850adbbf3&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:09.489862 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdefaa-fb3a-4025-84e5-6ae77bb82799 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:09.526152 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa177cec-164f-44f7-a82f-32496fb121db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:09.526976 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa177cec-164f-44f7-a82f-32496fb121db None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:09.527195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa177cec-164f-44f7-a82f-32496fb121db None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:09.527459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa177cec-164f-44f7-a82f-32496fb121db None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:09.528308 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa177cec-164f-44f7-a82f-32496fb121db None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:09.528614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa177cec-164f-44f7-a82f-32496fb121db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:09.530066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa177cec-164f-44f7-a82f-32496fb121db None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:09.548982 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 385/761] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jul 5 10:00:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:09.554413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb7999-a3fc-439f-8893-e2305dbde8f3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:09.606136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:09.662021 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae5f1c-8fe9-4eff-bfc1-f99bfba87c5d None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:09.662452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae5f1c-8fe9-4eff-bfc1-f99bfba87c5d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:09.662731 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae5f1c-8fe9-4eff-bfc1-f99bfba87c5d None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:09.664287 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 386/762] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:09.669875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea97c39-6640-4a43-a5f9-f300a9b6a4c7 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:09.670158 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea97c39-6640-4a43-a5f9-f300a9b6a4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:09.670467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea97c39-6640-4a43-a5f9-f300a9b6a4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:09.702897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-bea97c39-6640-4a43-a5f9-f300a9b6a4c7 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:09.704023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-bea97c39-6640-4a43-a5f9-f300a9b6a4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:09.737750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:09.740150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb7999-a3fc-439f-8893-e2305dbde8f3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:09.803579 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 387/763] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:09.833278 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea97c39-6640-4a43-a5f9-f300a9b6a4c7 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:09.848071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ee74-4f76-478a-af76-22ac81a07b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:09.848892 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ee74-4f76-478a-af76-22ac81a07b5b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:09.849179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ee74-4f76-478a-af76-22ac81a07b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:09.849458 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ee74-4f76-478a-af76-22ac81a07b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:09.875311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194ee74-4f76-478a-af76-22ac81a07b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:09.876773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194ee74-4f76-478a-af76-22ac81a07b5b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e194ee74-4f76-478a-af76-22ac81a07b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:09.922622 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:09.923596 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:09.924509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:09.924509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:09.924509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:09.924509 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:09.924509 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 377/764] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:09.929488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ee74-4f76-478a-af76-22ac81a07b5b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:09.947248 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea97c39-6640-4a43-a5f9-f300a9b6a4c7 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:09.986994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c96f82-3ad7-4912-90a2-9f1121e13d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:09.989247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c96f82-3ad7-4912-90a2-9f1121e13d59 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:09.991318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c96f82-3ad7-4912-90a2-9f1121e13d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:09.991797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c96f82-3ad7-4912-90a2-9f1121e13d59 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:09.992654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c96f82-3ad7-4912-90a2-9f1121e13d59 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:09.995134 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c96f82-3ad7-4912-90a2-9f1121e13d59 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-53c96f82-3ad7-4912-90a2-9f1121e13d59 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:10.029755 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:10.030786 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 10:00:10.031628 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:10.035832 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 388/765] 10.222.0.16 () {56 vars in 1129 bytes} [Tue Jul 5 10:00:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:10.041604 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c96f82-3ad7-4912-90a2-9f1121e13d59 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:10.074343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:10.101596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:10.138266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ee74-4f76-478a-af76-22ac81a07b5b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:10.166825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c96f82-3ad7-4912-90a2-9f1121e13d59 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:10.204404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68918c-4664-4949-8a85-420fa97efe1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:10.205559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68918c-4664-4949-8a85-420fa97efe1e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:10.206039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68918c-4664-4949-8a85-420fa97efe1e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:10.206527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68918c-4664-4949-8a85-420fa97efe1e None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:10.208189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68918c-4664-4949-8a85-420fa97efe1e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:10.208711 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68918c-4664-4949-8a85-420fa97efe1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:10.210343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68918c-4664-4949-8a85-420fa97efe1e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:10.214945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78411218-8f2b-4f7c-9bc9-c0bf8e5ed9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:10.215627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78411218-8f2b-4f7c-9bc9-c0bf8e5ed9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:10.216000 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78411218-8f2b-4f7c-9bc9-c0bf8e5ed9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:10.216188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78411218-8f2b-4f7c-9bc9-c0bf8e5ed9b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:10.216636 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 389/766] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jul 5 10:00:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:10.222276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78411218-8f2b-4f7c-9bc9-c0bf8e5ed9b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:10.222276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78411218-8f2b-4f7c-9bc9-c0bf8e5ed9b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:10.222276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78411218-8f2b-4f7c-9bc9-c0bf8e5ed9b2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:10.223538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68918c-4664-4949-8a85-420fa97efe1e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:10.237652 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 378/767] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:10.237652 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78411218-8f2b-4f7c-9bc9-c0bf8e5ed9b2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:10.335529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78411218-8f2b-4f7c-9bc9-c0bf8e5ed9b2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:10.347096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:10.369638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68918c-4664-4949-8a85-420fa97efe1e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:10.369877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168e973-5b89-4d7c-b0c2-786ee9e5325f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:10.370548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168e973-5b89-4d7c-b0c2-786ee9e5325f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:10.370836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168e973-5b89-4d7c-b0c2-786ee9e5325f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:10.371110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168e973-5b89-4d7c-b0c2-786ee9e5325f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:10.378137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168e973-5b89-4d7c-b0c2-786ee9e5325f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:10.379729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168e973-5b89-4d7c-b0c2-786ee9e5325f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:10.382552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-4168e973-5b89-4d7c-b0c2-786ee9e5325f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:10.399295 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:10.400246 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:10.401158 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:10.401158 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:10.401158 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:10.401158 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:10.401158 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:10.401158 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:10.401158 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:10.401158 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:10.405238 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 379/768] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:10.411233 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168e973-5b89-4d7c-b0c2-786ee9e5325f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:10.412046 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c83a02-0068-4b24-bd24-21874549dc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:10.412959 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83a02-0068-4b24-bd24-21874549dc6b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:10.413492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83a02-0068-4b24-bd24-21874549dc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:10.413971 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83a02-0068-4b24-bd24-21874549dc6b None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:10.414986 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c83a02-0068-4b24-bd24-21874549dc6b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:10.415492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c83a02-0068-4b24-bd24-21874549dc6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:10.417165 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c83a02-0068-4b24-bd24-21874549dc6b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:10.424363 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 390/769] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jul 5 10:00:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:10.429891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c83a02-0068-4b24-bd24-21874549dc6b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:10.546553 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168e973-5b89-4d7c-b0c2-786ee9e5325f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:10.557280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c83a02-0068-4b24-bd24-21874549dc6b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:10.598633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3b10e-2edc-4c45-bc97-5e129fa1074e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:10.599318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3b10e-2edc-4c45-bc97-5e129fa1074e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:10.599598 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3b10e-2edc-4c45-bc97-5e129fa1074e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:10.599869 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3b10e-2edc-4c45-bc97-5e129fa1074e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:10.600670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3b10e-2edc-4c45-bc97-5e129fa1074e None admin] RBAC: Adding query filter params (interface=public, service_id=c231ed272d6644889e93775e3351838b, region_id=RegionOne) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:10.600980 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3b10e-2edc-4c45-bc97-5e129fa1074e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:10.602409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3b10e-2edc-4c45-bc97-5e129fa1074e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:10.612691 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 391/770] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jul 5 10:00:10 2022] GET /identity/v3/endpoints?service_id=c231ed272d6644889e93775e3351838b&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:10.614448 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a91918-d1bb-4804-82aa-a309ff342b50 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:10.614739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a91918-d1bb-4804-82aa-a309ff342b50 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:10.615016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a91918-d1bb-4804-82aa-a309ff342b50 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:10.620606 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 392/771] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:10.621603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a91918-d1bb-4804-82aa-a309ff342b50 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:10.635710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:10.718088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e0265-1291-4bce-9e29-cb9d53c8c918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:10.719095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e0265-1291-4bce-9e29-cb9d53c8c918 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:10.719542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e0265-1291-4bce-9e29-cb9d53c8c918 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:10.719963 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e0265-1291-4bce-9e29-cb9d53c8c918 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:10.721031 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e0265-1291-4bce-9e29-cb9d53c8c918 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:10.721531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e0265-1291-4bce-9e29-cb9d53c8c918 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:10.723214 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e0265-1291-4bce-9e29-cb9d53c8c918 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:10.731860 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a91918-d1bb-4804-82aa-a309ff342b50 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:10.741454 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 380/772] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:10.743213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f24e72-37ea-42d6-843e-296cac379b2e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:10.743645 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f24e72-37ea-42d6-843e-296cac379b2e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:10.744078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f24e72-37ea-42d6-843e-296cac379b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:10.763302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-97f24e72-37ea-42d6-843e-296cac379b2e None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:10.764700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-97f24e72-37ea-42d6-843e-296cac379b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:10.772352 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47f496-1ec7-4c10-8f76-e20e6643d27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:10.773027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f496-1ec7-4c10-8f76-e20e6643d27f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:10.773308 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f496-1ec7-4c10-8f76-e20e6643d27f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:10.773580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f496-1ec7-4c10-8f76-e20e6643d27f None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:10.774864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47f496-1ec7-4c10-8f76-e20e6643d27f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:10.777137 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47f496-1ec7-4c10-8f76-e20e6643d27f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:10.783779 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 393/773] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jul 5 10:00:10 2022] GET /identity/v3/services => generated 772 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:10.784630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47f496-1ec7-4c10-8f76-e20e6643d27f None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:10.824900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:10.845595 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 381/774] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:10.871008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f24e72-37ea-42d6-843e-296cac379b2e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:10.879436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:10.945280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47f496-1ec7-4c10-8f76-e20e6643d27f None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:11.021231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f24e72-37ea-42d6-843e-296cac379b2e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:11.024456 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fe931-89f9-468e-9b7a-64da526ff101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:11.025361 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fe931-89f9-468e-9b7a-64da526ff101 None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:11.025640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fe931-89f9-468e-9b7a-64da526ff101 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:11.025914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fe931-89f9-468e-9b7a-64da526ff101 None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/fee6ee8f99c34aa281bf02ecd3f23216/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:11.030750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fe931-89f9-468e-9b7a-64da526ff101 None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=fee6ee8f99c34aa281bf02ecd3f23216, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:11.032802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fe931-89f9-468e-9b7a-64da526ff101 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:11.055554 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 394/775] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 10:00:10 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/fee6ee8f99c34aa281bf02ecd3f23216/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:00:11.077619 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bfd24-ae55-48d2-99ea-1a777ff44e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:11.078544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bfd24-ae55-48d2-99ea-1a777ff44e72 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:11.078898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bfd24-ae55-48d2-99ea-1a777ff44e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:11.079175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bfd24-ae55-48d2-99ea-1a777ff44e72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:11.092610 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56bfd24-ae55-48d2-99ea-1a777ff44e72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f56bfd24-ae55-48d2-99ea-1a777ff44e72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:11.100807 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:11.102316 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:11.103152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:11.107800 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 382/776] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:11.108868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fe931-89f9-468e-9b7a-64da526ff101 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:11.224712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fe931-89f9-468e-9b7a-64da526ff101 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:11.255495 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:11.261951 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2316f1-802e-4a3a-89c9-452b177d2731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:11.262669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2316f1-802e-4a3a-89c9-452b177d2731 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:11.262955 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2316f1-802e-4a3a-89c9-452b177d2731 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:11.263235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2316f1-802e-4a3a-89c9-452b177d2731 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:11.263990 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2316f1-802e-4a3a-89c9-452b177d2731 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:11.264297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2316f1-802e-4a3a-89c9-452b177d2731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:11.265746 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2316f1-802e-4a3a-89c9-452b177d2731 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:11.275496 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 395/777] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:00:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:11.281091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bfd24-ae55-48d2-99ea-1a777ff44e72 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:11.330097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99732556-a385-43e6-b07e-11ce66bbc5ab None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:11.330438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99732556-a385-43e6-b07e-11ce66bbc5ab None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:11.330625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99732556-a385-43e6-b07e-11ce66bbc5ab None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:11.332554 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 396/778] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:11.334456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:11.345263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26842af8-f9ab-42bd-9c94-646b4d2b8036 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:11.345544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26842af8-f9ab-42bd-9c94-646b4d2b8036 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:11.345821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26842af8-f9ab-42bd-9c94-646b4d2b8036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:11.364828 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-26842af8-f9ab-42bd-9c94-646b4d2b8036 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:11.366016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-26842af8-f9ab-42bd-9c94-646b4d2b8036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:11.393433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bfd24-ae55-48d2-99ea-1a777ff44e72 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:11.429698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779a47f-c649-4cd0-9a09-7c46819100be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:11.431033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779a47f-c649-4cd0-9a09-7c46819100be None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:11.431605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779a47f-c649-4cd0-9a09-7c46819100be None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:11.432069 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779a47f-c649-4cd0-9a09-7c46819100be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:11.436009 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 397/779] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:11.443337 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779a47f-c649-4cd0-9a09-7c46819100be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:11.445079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779a47f-c649-4cd0-9a09-7c46819100be None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:11.458964 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26842af8-f9ab-42bd-9c94-646b4d2b8036 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e779a47f-c649-4cd0-9a09-7c46819100be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:11.463751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:11.464601 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:11.465472 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:11.465472 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:11.465472 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:11.465472 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:11.467646 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 383/780] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:11.473375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779a47f-c649-4cd0-9a09-7c46819100be None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:11.602267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26842af8-f9ab-42bd-9c94-646b4d2b8036 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:11.631423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:11.649638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ccd9c-a894-49f3-8088-71d6b1f14953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:11.650184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:11.651543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ccd9c-a894-49f3-8088-71d6b1f14953 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:11.652727 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ccd9c-a894-49f3-8088-71d6b1f14953 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:11.653905 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ccd9c-a894-49f3-8088-71d6b1f14953 None admin] PATH_INFO: `/v3/services/network` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:11.655276 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ccd9c-a894-49f3-8088-71d6b1f14953 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:11.656864 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779a47f-c649-4cd0-9a09-7c46819100be None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:11.658452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ccd9c-a894-49f3-8088-71d6b1f14953 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-f82ccd9c-a894-49f3-8088-71d6b1f14953 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:11.677478 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:11.678731 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 10:00:11.679797 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:11.692227 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 398/781] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:11 2022] GET /identity/v3/services/network => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:11.694853 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ccd9c-a894-49f3-8088-71d6b1f14953 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:11.736355 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a347041-ace2-4afa-aa12-01719fcb445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:11.737402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a347041-ace2-4afa-aa12-01719fcb445f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:11.737402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a347041-ace2-4afa-aa12-01719fcb445f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:11.737673 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a347041-ace2-4afa-aa12-01719fcb445f None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:11.738561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a347041-ace2-4afa-aa12-01719fcb445f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:11.738876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a347041-ace2-4afa-aa12-01719fcb445f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:11.740265 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a347041-ace2-4afa-aa12-01719fcb445f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:11.764477 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 384/782] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:11.773894 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a347041-ace2-4afa-aa12-01719fcb445f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:11.809010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ccd9c-a894-49f3-8088-71d6b1f14953 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:11.859037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ba7ec-4383-4313-8cf4-f672e8c82584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:11.859647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ba7ec-4383-4313-8cf4-f672e8c82584 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:11.859931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ba7ec-4383-4313-8cf4-f672e8c82584 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:11.860207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ba7ec-4383-4313-8cf4-f672e8c82584 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:11.860921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ba7ec-4383-4313-8cf4-f672e8c82584 None admin] RBAC: Adding query filter params (name=network) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:11.861224 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ba7ec-4383-4313-8cf4-f672e8c82584 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:11.862621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ba7ec-4383-4313-8cf4-f672e8c82584 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:11.876186 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 399/783] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:11.881650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ba7ec-4383-4313-8cf4-f672e8c82584 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:11.903406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a347041-ace2-4afa-aa12-01719fcb445f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:11.940746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee813aff-0488-4272-91ec-ac141d5cac62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:11.941646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee813aff-0488-4272-91ec-ac141d5cac62 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:11.941904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee813aff-0488-4272-91ec-ac141d5cac62 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:11.942172 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee813aff-0488-4272-91ec-ac141d5cac62 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:11.956079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee813aff-0488-4272-91ec-ac141d5cac62 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:11.957911 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee813aff-0488-4272-91ec-ac141d5cac62 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ee813aff-0488-4272-91ec-ac141d5cac62 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:11.973380 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:11.974413 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:11.975334 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:11.975334 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:11.975334 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:11.975334 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:11.975334 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:11.975334 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:11.975334 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:11.975334 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:11.975334 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 10:00:11.975334 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:11.975334 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 385/784] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:11.981026 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d023b-64e6-43bd-b3b9-8a823e2045ea None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:11.981026 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d023b-64e6-43bd-b3b9-8a823e2045ea None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:11.981026 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d023b-64e6-43bd-b3b9-8a823e2045ea None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:11.981026 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 386/785] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:11.986549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d023b-64e6-43bd-b3b9-8a823e2045ea None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:12.010642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:12.014591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:12.036649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ba7ec-4383-4313-8cf4-f672e8c82584 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:12.071691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc736e-47be-491d-bd75-020a019887a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:12.072356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc736e-47be-491d-bd75-020a019887a4 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:12.072629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc736e-47be-491d-bd75-020a019887a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:12.072898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc736e-47be-491d-bd75-020a019887a4 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:12.073595 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc736e-47be-491d-bd75-020a019887a4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:12.073911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc736e-47be-491d-bd75-020a019887a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:12.075367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc736e-47be-491d-bd75-020a019887a4 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:12.080998 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 400/786] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:12.082580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12c086-3465-4cf3-b790-f15c95012114 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:12.082893 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12c086-3465-4cf3-b790-f15c95012114 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:12.083171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12c086-3465-4cf3-b790-f15c95012114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:12.100928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-5a12c086-3465-4cf3-b790-f15c95012114 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:12.102106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5a12c086-3465-4cf3-b790-f15c95012114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:12.124339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d023b-64e6-43bd-b3b9-8a823e2045ea None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:12.160999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c602b3c-3326-4808-9d75-f4988d6f9028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:12.161622 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c602b3c-3326-4808-9d75-f4988d6f9028 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:12.161948 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c602b3c-3326-4808-9d75-f4988d6f9028 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:12.162244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c602b3c-3326-4808-9d75-f4988d6f9028 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:12.163102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c602b3c-3326-4808-9d75-f4988d6f9028 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:12.163460 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c602b3c-3326-4808-9d75-f4988d6f9028 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:12.164936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c602b3c-3326-4808-9d75-f4988d6f9028 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:12.188156 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 387/787] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jul 5 10:00:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:12.189024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c602b3c-3326-4808-9d75-f4988d6f9028 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:12.198677 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 401/788] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:12.201351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12c086-3465-4cf3-b790-f15c95012114 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:12.267474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:12.281876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:12.308278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c602b3c-3326-4808-9d75-f4988d6f9028 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:12.341640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12c086-3465-4cf3-b790-f15c95012114 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:12.345036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04527e5-d571-4de7-92b0-6f5c0f56c99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:12.345563 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04527e5-d571-4de7-92b0-6f5c0f56c99c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:12.345847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04527e5-d571-4de7-92b0-6f5c0f56c99c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:12.346124 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04527e5-d571-4de7-92b0-6f5c0f56c99c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:12.346958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04527e5-d571-4de7-92b0-6f5c0f56c99c None admin] RBAC: Adding query filter params (interface=public, service_id=2fcbd71d615b4072b3caff139e706f8b, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:12.347277 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04527e5-d571-4de7-92b0-6f5c0f56c99c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:12.348742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04527e5-d571-4de7-92b0-6f5c0f56c99c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:12.361044 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 388/789] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jul 5 10:00:12 2022] GET /identity/v3/endpoints?service_id=2fcbd71d615b4072b3caff139e706f8b&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:12.361934 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04527e5-d571-4de7-92b0-6f5c0f56c99c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:12.385138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf1d59-8669-407c-89e5-acc75451f79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:12.386608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf1d59-8669-407c-89e5-acc75451f79e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:12.387410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf1d59-8669-407c-89e5-acc75451f79e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:12.388359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf1d59-8669-407c-89e5-acc75451f79e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:12.403615 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf1d59-8669-407c-89e5-acc75451f79e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:12.406054 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf1d59-8669-407c-89e5-acc75451f79e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-7eaf1d59-8669-407c-89e5-acc75451f79e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:12.429920 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:12.430936 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:12.431831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:12.431831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:12.431831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:12.431831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:12.434299 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 402/790] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:12.435330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf1d59-8669-407c-89e5-acc75451f79e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:12.485758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04527e5-d571-4de7-92b0-6f5c0f56c99c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:12.531692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3814a98-e53b-49e5-b6e6-8540bdc8e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:12.533498 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3814a98-e53b-49e5-b6e6-8540bdc8e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:12.534002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3814a98-e53b-49e5-b6e6-8540bdc8e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:12.534505 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3814a98-e53b-49e5-b6e6-8540bdc8e5a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:12.544690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:12.547030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:12.549384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf1d59-8669-407c-89e5-acc75451f79e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:12.550931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3814a98-e53b-49e5-b6e6-8540bdc8e5a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a3814a98-e53b-49e5-b6e6-8540bdc8e5a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:12.561751 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:12.562881 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:00:12.563930 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:12.571668 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 389/791] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 10:00:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:12.571668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3814a98-e53b-49e5-b6e6-8540bdc8e5a1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:12.601311 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ddc0-d503-415a-bb0d-584a761b8062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:12.602581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ddc0-d503-415a-bb0d-584a761b8062 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:12.603635 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ddc0-d503-415a-bb0d-584a761b8062 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:12.604445 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ddc0-d503-415a-bb0d-584a761b8062 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:12.605547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2ddc0-d503-415a-bb0d-584a761b8062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:12.607198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2ddc0-d503-415a-bb0d-584a761b8062 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:12.614642 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 403/792] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jul 5 10:00:12 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:12.615706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ddc0-d503-415a-bb0d-584a761b8062 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:12.738921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3814a98-e53b-49e5-b6e6-8540bdc8e5a1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:12.779830 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f05259-6817-4b13-8039-8ef8d0b60615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:12.780855 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f05259-6817-4b13-8039-8ef8d0b60615 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:12.781823 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f05259-6817-4b13-8039-8ef8d0b60615 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:12.782418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f05259-6817-4b13-8039-8ef8d0b60615 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:12.783428 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f05259-6817-4b13-8039-8ef8d0b60615 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:12.783888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f05259-6817-4b13-8039-8ef8d0b60615 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:12.785389 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ddc0-d503-415a-bb0d-584a761b8062 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:12.785717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f05259-6817-4b13-8039-8ef8d0b60615 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:12.801077 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 390/793] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:12.808671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f05259-6817-4b13-8039-8ef8d0b60615 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:12.859629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1f17b-d437-4572-94c1-a9c8b738330b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:12.861709 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f17b-d437-4572-94c1-a9c8b738330b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:12.861709 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f17b-d437-4572-94c1-a9c8b738330b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:12.861709 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f17b-d437-4572-94c1-a9c8b738330b None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:12.861709 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1f17b-d437-4572-94c1-a9c8b738330b None admin] RBAC: Adding query filter params (name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:12.862009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1f17b-d437-4572-94c1-a9c8b738330b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:12.863923 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1f17b-d437-4572-94c1-a9c8b738330b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:12.872997 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 404/794] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 10:00:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:12.878248 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1f17b-d437-4572-94c1-a9c8b738330b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:12.908847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:12.916484 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:12.962856 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f05259-6817-4b13-8039-8ef8d0b60615 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:13.001928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1f17b-d437-4572-94c1-a9c8b738330b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:13.017864 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16316cc8-184f-480f-8ae8-6d880f232901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:13.018707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16316cc8-184f-480f-8ae8-6d880f232901 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.018991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16316cc8-184f-480f-8ae8-6d880f232901 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.019270 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16316cc8-184f-480f-8ae8-6d880f232901 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.026828 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16316cc8-184f-480f-8ae8-6d880f232901 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:13.028533 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16316cc8-184f-480f-8ae8-6d880f232901 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-16316cc8-184f-480f-8ae8-6d880f232901 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:13.048804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:13.049753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:13.051074 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:13.051074 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:13.051074 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:13.051074 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:13.051074 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:13.051074 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:13.051074 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:13.051074 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:13.054044 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afd11b-11e5-454a-a7aa-8bb3f5447b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:13.055151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afd11b-11e5-454a-a7aa-8bb3f5447b04 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.055631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afd11b-11e5-454a-a7aa-8bb3f5447b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.056107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afd11b-11e5-454a-a7aa-8bb3f5447b04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.058769 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 391/795] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:13.075071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afd11b-11e5-454a-a7aa-8bb3f5447b04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:13.076880 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afd11b-11e5-454a-a7aa-8bb3f5447b04 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:13.087862 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16316cc8-184f-480f-8ae8-6d880f232901 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-d5afd11b-11e5-454a-a7aa-8bb3f5447b04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:13.112634 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:13.113669 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:13.114626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:13.114626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:13.114626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:13.114626 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:13.117394 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 405/796] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:13.123883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afd11b-11e5-454a-a7aa-8bb3f5447b04 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:13.261534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:13.263208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16316cc8-184f-480f-8ae8-6d880f232901 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:13.283247 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:13.290477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afd11b-11e5-454a-a7aa-8bb3f5447b04 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:13.297639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ae86d-7cf9-4883-bd32-60efe61771e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:13.298340 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae86d-7cf9-4883-bd32-60efe61771e3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.298632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae86d-7cf9-4883-bd32-60efe61771e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.298912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae86d-7cf9-4883-bd32-60efe61771e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.299755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ae86d-7cf9-4883-bd32-60efe61771e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:13.300009 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ae86d-7cf9-4883-bd32-60efe61771e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:13.301508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ae86d-7cf9-4883-bd32-60efe61771e3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:13.315590 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 392/797] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:13.321364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f4140-2621-491e-9d3b-1defaa69650e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.321364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f4140-2621-491e-9d3b-1defaa69650e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.321364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f4140-2621-491e-9d3b-1defaa69650e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.323454 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 393/798] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:13.324090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f4140-2621-491e-9d3b-1defaa69650e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:13.335657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24516152-ecb3-438f-aab8-1e13ae882489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:13.336759 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24516152-ecb3-438f-aab8-1e13ae882489 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.337241 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24516152-ecb3-438f-aab8-1e13ae882489 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.337733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24516152-ecb3-438f-aab8-1e13ae882489 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.338809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24516152-ecb3-438f-aab8-1e13ae882489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:13.339333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24516152-ecb3-438f-aab8-1e13ae882489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:13.341182 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24516152-ecb3-438f-aab8-1e13ae882489 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:13.355502 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 406/799] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:13.357656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0854e-7c97-482b-9971-6cb9b24f9b20 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.358106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0854e-7c97-482b-9971-6cb9b24f9b20 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.358581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0854e-7c97-482b-9971-6cb9b24f9b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.376325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-47b0854e-7c97-482b-9971-6cb9b24f9b20 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:13.377532 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-47b0854e-7c97-482b-9971-6cb9b24f9b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:13.428743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f4140-2621-491e-9d3b-1defaa69650e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:13.451037 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 407/800] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:13.452647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ce108-4991-4a34-a2e1-7f396bd4f7c1 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.452932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ce108-4991-4a34-a2e1-7f396bd4f7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.453204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ce108-4991-4a34-a2e1-7f396bd4f7c1 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.454740 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 408/801] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:13.455423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ce108-4991-4a34-a2e1-7f396bd4f7c1 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:13.470766 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251ddce-4f22-41ca-ae74-33ece0bca1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:13.472004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251ddce-4f22-41ca-ae74-33ece0bca1db None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.472528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251ddce-4f22-41ca-ae74-33ece0bca1db None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.473024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251ddce-4f22-41ca-ae74-33ece0bca1db None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/dbfded29b70140a387f7b8c07c296861/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.475339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251ddce-4f22-41ca-ae74-33ece0bca1db None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=dbfded29b70140a387f7b8c07c296861, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:13.477547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251ddce-4f22-41ca-ae74-33ece0bca1db None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:13.494283 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 394/802] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 10:00:13 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/dbfded29b70140a387f7b8c07c296861/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:00:13.494283 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fb018-4733-48c5-9ca1-4123d488bc63 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.494283 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fb018-4733-48c5-9ca1-4123d488bc63 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.494606 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fb018-4733-48c5-9ca1-4123d488bc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.522501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1e6fb018-4733-48c5-9ca1-4123d488bc63 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:13.524803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1e6fb018-4733-48c5-9ca1-4123d488bc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:13.542692 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:13.582895 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:13.584093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ce108-4991-4a34-a2e1-7f396bd4f7c1 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:13.635420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59aa17-72a0-4cf4-b98f-33fd8c7f44d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:13.636065 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 395/803] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:13.636842 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fb018-4733-48c5-9ca1-4123d488bc63 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:13.637850 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59aa17-72a0-4cf4-b98f-33fd8c7f44d2 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.638195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59aa17-72a0-4cf4-b98f-33fd8c7f44d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.638424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59aa17-72a0-4cf4-b98f-33fd8c7f44d2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.653411 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59aa17-72a0-4cf4-b98f-33fd8c7f44d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:13.655004 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59aa17-72a0-4cf4-b98f-33fd8c7f44d2 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-1a59aa17-72a0-4cf4-b98f-33fd8c7f44d2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:13.674613 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:13.675689 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:13.676621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:13.676621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:13.676621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:13.676621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:13.676621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:13.676621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:13.676621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:13.676621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:13.676621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:13.676621 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:13.676621 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 409/804] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jul 5 10:00:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:13.677199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59aa17-72a0-4cf4-b98f-33fd8c7f44d2 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:13.769210 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fb018-4733-48c5-9ca1-4123d488bc63 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:13.806198 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a3b4c-fa1c-42ae-9b7c-39275f9fe716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:13.806970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a3b4c-fa1c-42ae-9b7c-39275f9fe716 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.807278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a3b4c-fa1c-42ae-9b7c-39275f9fe716 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.807534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a3b4c-fa1c-42ae-9b7c-39275f9fe716 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.808200 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a3b4c-fa1c-42ae-9b7c-39275f9fe716 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:13.809769 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a3b4c-fa1c-42ae-9b7c-39275f9fe716 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:13.816769 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59aa17-72a0-4cf4-b98f-33fd8c7f44d2 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-559a3b4c-fa1c-42ae-9b7c-39275f9fe716 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:13.823062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:13.823999 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:13.824884 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 396/805] 10.222.0.16 () {56 vars in 1129 bytes} [Tue Jul 5 10:00:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:13.825725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a3b4c-fa1c-42ae-9b7c-39275f9fe716 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:13.851925 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:13.862534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2cb5b-d685-4b69-9c8e-1ce4d6c08900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:13.863380 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2cb5b-d685-4b69-9c8e-1ce4d6c08900 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:13.863864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2cb5b-d685-4b69-9c8e-1ce4d6c08900 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:13.864005 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2cb5b-d685-4b69-9c8e-1ce4d6c08900 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:13.874529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:13.874673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2cb5b-d685-4b69-9c8e-1ce4d6c08900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-2cf2cb5b-d685-4b69-9c8e-1ce4d6c08900 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:13.883411 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:13.884602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:13.885530 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:13.889882 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 410/806] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:13.890893 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2cb5b-d685-4b69-9c8e-1ce4d6c08900 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:13.955992 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a3b4c-fa1c-42ae-9b7c-39275f9fe716 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:13.996368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2cb5b-d685-4b69-9c8e-1ce4d6c08900 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:14.005221 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed60fff-32fb-4ad1-adcd-6a89cae216e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.005813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed60fff-32fb-4ad1-adcd-6a89cae216e4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.006101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed60fff-32fb-4ad1-adcd-6a89cae216e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.006427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed60fff-32fb-4ad1-adcd-6a89cae216e4 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.007280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed60fff-32fb-4ad1-adcd-6a89cae216e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:14.007593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed60fff-32fb-4ad1-adcd-6a89cae216e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.009039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed60fff-32fb-4ad1-adcd-6a89cae216e4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.027180 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 397/807] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jul 5 10:00:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:14.028127 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed60fff-32fb-4ad1-adcd-6a89cae216e4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:14.041309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e64a31-e332-44c3-8dd7-7b15e5b53bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.042327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e64a31-e332-44c3-8dd7-7b15e5b53bda None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.043060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e64a31-e332-44c3-8dd7-7b15e5b53bda None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.043996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e64a31-e332-44c3-8dd7-7b15e5b53bda None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.045064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e64a31-e332-44c3-8dd7-7b15e5b53bda None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:14.045701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e64a31-e332-44c3-8dd7-7b15e5b53bda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.048103 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e64a31-e332-44c3-8dd7-7b15e5b53bda None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.055940 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 411/808] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jul 5 10:00:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:14.058346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e64a31-e332-44c3-8dd7-7b15e5b53bda None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:14.116202 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:14.149356 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:14.159087 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed60fff-32fb-4ad1-adcd-6a89cae216e4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:14.180020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e64a31-e332-44c3-8dd7-7b15e5b53bda None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:14.197840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b601dfa-a3b0-419d-a033-f79a9426c40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.199224 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b601dfa-a3b0-419d-a033-f79a9426c40f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.199711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b601dfa-a3b0-419d-a033-f79a9426c40f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.200186 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b601dfa-a3b0-419d-a033-f79a9426c40f None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.201134 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b601dfa-a3b0-419d-a033-f79a9426c40f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:14.202141 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b601dfa-a3b0-419d-a033-f79a9426c40f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.204260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b601dfa-a3b0-419d-a033-f79a9426c40f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.216069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d72b8-7cd2-403e-a5c3-2dd2778b1ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.216416 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 398/809] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:00:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:14.216975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d72b8-7cd2-403e-a5c3-2dd2778b1ffd None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.217254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d72b8-7cd2-403e-a5c3-2dd2778b1ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.217565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d72b8-7cd2-403e-a5c3-2dd2778b1ffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.218163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b601dfa-a3b0-419d-a033-f79a9426c40f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:14.229912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d72b8-7cd2-403e-a5c3-2dd2778b1ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.231462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d72b8-7cd2-403e-a5c3-2dd2778b1ffd None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-cd6d72b8-7cd2-403e-a5c3-2dd2778b1ffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:14.249300 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:14.250190 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:14.251709 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:14.251709 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:14.251709 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:14.251709 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:14.251709 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 412/810] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:14.251709 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d72b8-7cd2-403e-a5c3-2dd2778b1ffd None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:14.326585 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b601dfa-a3b0-419d-a033-f79a9426c40f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:14.356019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d72b8-7cd2-403e-a5c3-2dd2778b1ffd None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:14.357534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:14.362822 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303718e-9c75-4909-b495-9f8400b493f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.363796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303718e-9c75-4909-b495-9f8400b493f9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.364280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303718e-9c75-4909-b495-9f8400b493f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.364779 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303718e-9c75-4909-b495-9f8400b493f9 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.365684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303718e-9c75-4909-b495-9f8400b493f9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:14.366189 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303718e-9c75-4909-b495-9f8400b493f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.367773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303718e-9c75-4909-b495-9f8400b493f9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.373434 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 399/811] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jul 5 10:00:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:14.374597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303718e-9c75-4909-b495-9f8400b493f9 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:14.392176 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8d531-26e3-408f-a646-2d8ef2ba6795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.393033 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8d531-26e3-408f-a646-2d8ef2ba6795 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.393373 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8d531-26e3-408f-a646-2d8ef2ba6795 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.393656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8d531-26e3-408f-a646-2d8ef2ba6795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.395205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:14.405018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8d531-26e3-408f-a646-2d8ef2ba6795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.406606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8d531-26e3-408f-a646-2d8ef2ba6795 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-97c8d531-26e3-408f-a646-2d8ef2ba6795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:14.424280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:14.425203 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:14.426727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:14.426727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:14.426727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:14.426727 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:14.426727 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 413/812] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:14.427039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8d531-26e3-408f-a646-2d8ef2ba6795 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:14.484996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303718e-9c75-4909-b495-9f8400b493f9 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:14.521661 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4dc0f8-bb84-4a15-ac37-21b808020ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.522812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dc0f8-bb84-4a15-ac37-21b808020ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.523298 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dc0f8-bb84-4a15-ac37-21b808020ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.523790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dc0f8-bb84-4a15-ac37-21b808020ad3 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.524810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4dc0f8-bb84-4a15-ac37-21b808020ad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:14.525359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4dc0f8-bb84-4a15-ac37-21b808020ad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.527775 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4dc0f8-bb84-4a15-ac37-21b808020ad3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.530924 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8d531-26e3-408f-a646-2d8ef2ba6795 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:14.541374 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 400/813] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:14.542713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4dc0f8-bb84-4a15-ac37-21b808020ad3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:14.566016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b82a4-7c0d-4bb3-9450-45fc38a0e404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.566773 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b82a4-7c0d-4bb3-9450-45fc38a0e404 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.567062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b82a4-7c0d-4bb3-9450-45fc38a0e404 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.567362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b82a4-7c0d-4bb3-9450-45fc38a0e404 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.568211 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b82a4-7c0d-4bb3-9450-45fc38a0e404 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.569771 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b82a4-7c0d-4bb3-9450-45fc38a0e404 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.594972 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 414/814] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jul 5 10:00:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:14.595867 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b82a4-7c0d-4bb3-9450-45fc38a0e404 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:14.610060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:14.638061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:14.660998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4dc0f8-bb84-4a15-ac37-21b808020ad3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:14.749213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b82a4-7c0d-4bb3-9450-45fc38a0e404 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:14.752189 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e4d51-c1a0-4abc-85ba-91a039bc767b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.753148 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e4d51-c1a0-4abc-85ba-91a039bc767b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.753674 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e4d51-c1a0-4abc-85ba-91a039bc767b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.754158 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e4d51-c1a0-4abc-85ba-91a039bc767b None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.755135 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e4d51-c1a0-4abc-85ba-91a039bc767b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:14.755740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e4d51-c1a0-4abc-85ba-91a039bc767b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.757399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e4d51-c1a0-4abc-85ba-91a039bc767b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.787409 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 401/815] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:14.790492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9065c-a68f-4707-bc79-283efad1edad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.791309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9065c-a68f-4707-bc79-283efad1edad None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.791653 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9065c-a68f-4707-bc79-283efad1edad None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.791930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9065c-a68f-4707-bc79-283efad1edad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.792806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e4d51-c1a0-4abc-85ba-91a039bc767b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:14.803017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9065c-a68f-4707-bc79-283efad1edad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.804506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9065c-a68f-4707-bc79-283efad1edad None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-38f9065c-a68f-4707-bc79-283efad1edad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:14.820237 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:14.821188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:14.822093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:14.822093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:14.822093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:14.822093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:14.822093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:14.822093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:14.822093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:14.822093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:14.822093 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 415/816] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:14.827281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9065c-a68f-4707-bc79-283efad1edad None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:14.900748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:14.925972 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e4d51-c1a0-4abc-85ba-91a039bc767b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:14.937363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9065c-a68f-4707-bc79-283efad1edad None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:14.970246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c3acd-4bb8-47d4-a23c-f20c293b7702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.971065 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c3acd-4bb8-47d4-a23c-f20c293b7702 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.971351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c3acd-4bb8-47d4-a23c-f20c293b7702 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.971625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c3acd-4bb8-47d4-a23c-f20c293b7702 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.972175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680fa8f-c390-4a66-b1c9-b30c8c40ef24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:14.972822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680fa8f-c390-4a66-b1c9-b30c8c40ef24 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:14.973111 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680fa8f-c390-4a66-b1c9-b30c8c40ef24 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:14.973388 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680fa8f-c390-4a66-b1c9-b30c8c40ef24 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:14.974182 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680fa8f-c390-4a66-b1c9-b30c8c40ef24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:14.974522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680fa8f-c390-4a66-b1c9-b30c8c40ef24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.975039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:14.975965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680fa8f-c390-4a66-b1c9-b30c8c40ef24 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.987405 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c3acd-4bb8-47d4-a23c-f20c293b7702 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:14.988879 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 416/817] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:14.988998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c3acd-4bb8-47d4-a23c-f20c293b7702 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:14.994685 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680fa8f-c390-4a66-b1c9-b30c8c40ef24 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e26c3acd-4bb8-47d4-a23c-f20c293b7702 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:15.009513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:15.010542 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:15.011491 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:15.011491 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:15.011491 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:15.011491 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:15.011491 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:15.011491 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:15.011491 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:15.011491 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:15.011491 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:15.011491 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:15.017426 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 402/818] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jul 5 10:00:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:15.023423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c3acd-4bb8-47d4-a23c-f20c293b7702 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:15.101603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680fa8f-c390-4a66-b1c9-b30c8c40ef24 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:15.143488 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:15.147012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c3acd-4bb8-47d4-a23c-f20c293b7702 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:15.150843 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d9d59-6cd5-4a80-8b54-15a45a682099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:15.151687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d9d59-6cd5-4a80-8b54-15a45a682099 None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:15.151971 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d9d59-6cd5-4a80-8b54-15a45a682099 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:15.154600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d9d59-6cd5-4a80-8b54-15a45a682099 None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/84a8d62afe964cb49acc236850adbbf3/roles/1821d382b71242ff8f94e4930b6ab71d` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:15.173975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d9d59-6cd5-4a80-8b54-15a45a682099 None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=84a8d62afe964cb49acc236850adbbf3, role_id=1821d382b71242ff8f94e4930b6ab71d)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:15.177019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d9d59-6cd5-4a80-8b54-15a45a682099 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:15.183651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f32b89-de57-445c-ba55-6e6c111ce517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:15.184334 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f32b89-de57-445c-ba55-6e6c111ce517 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:15.184656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f32b89-de57-445c-ba55-6e6c111ce517 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:15.184940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f32b89-de57-445c-ba55-6e6c111ce517 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:15.185956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f32b89-de57-445c-ba55-6e6c111ce517 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:15.186382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f32b89-de57-445c-ba55-6e6c111ce517 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:15.187938 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f32b89-de57-445c-ba55-6e6c111ce517 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:15.194950 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 417/819] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 10:00:14 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/84a8d62afe964cb49acc236850adbbf3/roles/1821d382b71242ff8f94e4930b6ab71d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:00:15.196778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8266a06-278c-4d00-a031-77fb5782b944 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:15.197313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8266a06-278c-4d00-a031-77fb5782b944 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:15.197788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8266a06-278c-4d00-a031-77fb5782b944 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:15.203881 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 418/820] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:15.206273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4f65b-3917-4134-870f-6afabe58006d None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:15.206869 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4f65b-3917-4134-870f-6afabe58006d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:15.207714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4f65b-3917-4134-870f-6afabe58006d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:15.227522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-42e4f65b-3917-4134-870f-6afabe58006d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:15.233315 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 403/821] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jul 5 10:00:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:15.234162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-42e4f65b-3917-4134-870f-6afabe58006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:15.242267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f32b89-de57-445c-ba55-6e6c111ce517 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:15.315890 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 419/822] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:15.342054 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4f65b-3917-4134-870f-6afabe58006d None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:15.370977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:15.447813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f32b89-de57-445c-ba55-6e6c111ce517 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:15.494387 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1aaab-58a4-4190-a7ea-224fb8556730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:15.495078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1aaab-58a4-4190-a7ea-224fb8556730 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:15.495368 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1aaab-58a4-4190-a7ea-224fb8556730 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:15.495649 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1aaab-58a4-4190-a7ea-224fb8556730 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:15.606686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4f65b-3917-4134-870f-6afabe58006d None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:15.614100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:15.617481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1aaab-58a4-4190-a7ea-224fb8556730 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:15.619143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1aaab-58a4-4190-a7ea-224fb8556730 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:15.644032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4881514-d429-457a-80de-b89a2b1d6e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:15.644964 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4881514-d429-457a-80de-b89a2b1d6e17 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:15.645264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4881514-d429-457a-80de-b89a2b1d6e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:15.645543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4881514-d429-457a-80de-b89a2b1d6e17 None admin] PATH_INFO: `/v3/services/network` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:15.646258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4881514-d429-457a-80de-b89a2b1d6e17 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:15.648119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4881514-d429-457a-80de-b89a2b1d6e17 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-d4881514-d429-457a-80de-b89a2b1d6e17 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:15.662264 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:15.663366 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:15.664449 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 420/823] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:15 2022] GET /identity/v3/services/network => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:15.672340 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4881514-d429-457a-80de-b89a2b1d6e17 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:15.721051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:15.785691 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 404/824] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:00:15.787732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442685bd-11f1-4ac2-9c19-46dd30ea0dc4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:15.788032 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442685bd-11f1-4ac2-9c19-46dd30ea0dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:15.788312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442685bd-11f1-4ac2-9c19-46dd30ea0dc4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:15.793632 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 405/825] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:15.793632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442685bd-11f1-4ac2-9c19-46dd30ea0dc4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:15.807741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4881514-d429-457a-80de-b89a2b1d6e17 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:15.843119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b96521-1df9-4a4e-b35d-ff8d70ff1308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:15.844215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b96521-1df9-4a4e-b35d-ff8d70ff1308 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:15.844691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b96521-1df9-4a4e-b35d-ff8d70ff1308 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:15.845214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b96521-1df9-4a4e-b35d-ff8d70ff1308 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:15.846109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b96521-1df9-4a4e-b35d-ff8d70ff1308 None admin] RBAC: Adding query filter params (name=network) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:15.846643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b96521-1df9-4a4e-b35d-ff8d70ff1308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:15.848351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b96521-1df9-4a4e-b35d-ff8d70ff1308 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:15.853709 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 421/826] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:15.855282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98108b04-c303-449c-a950-541cbc1eb0c7 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:15.855769 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98108b04-c303-449c-a950-541cbc1eb0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:15.856231 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98108b04-c303-449c-a950-541cbc1eb0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:15.872897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-98108b04-c303-449c-a950-541cbc1eb0c7 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:15.874487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-98108b04-c303-449c-a950-541cbc1eb0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:15.902428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:15.906871 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442685bd-11f1-4ac2-9c19-46dd30ea0dc4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:15.942867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995ffb9-a38b-4d46-9e5c-d8dda16501ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:15.943680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995ffb9-a38b-4d46-9e5c-d8dda16501ec None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:15.943978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995ffb9-a38b-4d46-9e5c-d8dda16501ec None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:15.944247 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995ffb9-a38b-4d46-9e5c-d8dda16501ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:15.954543 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 422/827] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:15.954988 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995ffb9-a38b-4d46-9e5c-d8dda16501ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:15.955914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98108b04-c303-449c-a950-541cbc1eb0c7 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:15.956458 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995ffb9-a38b-4d46-9e5c-d8dda16501ec None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-3995ffb9-a38b-4d46-9e5c-d8dda16501ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:15.978321 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:15.979276 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:15.980161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:15.980161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:15.980161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:15.980161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:15.980161 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 406/828] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:15.981766 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995ffb9-a38b-4d46-9e5c-d8dda16501ec None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:15.988507 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:16.075083 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98108b04-c303-449c-a950-541cbc1eb0c7 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:16.092772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995ffb9-a38b-4d46-9e5c-d8dda16501ec None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:16.111056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df642ec1-01fe-4674-b12d-7bc24c86ed5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:16.112127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df642ec1-01fe-4674-b12d-7bc24c86ed5b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:16.112621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df642ec1-01fe-4674-b12d-7bc24c86ed5b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:16.113143 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df642ec1-01fe-4674-b12d-7bc24c86ed5b None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:16.114104 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df642ec1-01fe-4674-b12d-7bc24c86ed5b None admin] RBAC: Adding query filter params (type=network) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:16.114668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df642ec1-01fe-4674-b12d-7bc24c86ed5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:16.116324 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df642ec1-01fe-4674-b12d-7bc24c86ed5b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:16.122560 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 423/829] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:16.124414 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df642ec1-01fe-4674-b12d-7bc24c86ed5b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:16.128841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4a068-9ed6-41c3-a99a-43ea40124651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:16.129489 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4a068-9ed6-41c3-a99a-43ea40124651 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:16.130506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4a068-9ed6-41c3-a99a-43ea40124651 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:16.130506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4a068-9ed6-41c3-a99a-43ea40124651 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:16.130852 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4a068-9ed6-41c3-a99a-43ea40124651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:16.131408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4a068-9ed6-41c3-a99a-43ea40124651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:16.133230 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4a068-9ed6-41c3-a99a-43ea40124651 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:16.151340 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 407/830] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:16.152513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4a068-9ed6-41c3-a99a-43ea40124651 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:16.159568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:16.218409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:16.234806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df642ec1-01fe-4674-b12d-7bc24c86ed5b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:16.258443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4a068-9ed6-41c3-a99a-43ea40124651 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:16.269797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934ee80-85d0-4b96-b0ab-39fec1295a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:16.270878 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934ee80-85d0-4b96-b0ab-39fec1295a40 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:16.271376 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934ee80-85d0-4b96-b0ab-39fec1295a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:16.271846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934ee80-85d0-4b96-b0ab-39fec1295a40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:16.282442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934ee80-85d0-4b96-b0ab-39fec1295a40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-d934ee80-85d0-4b96-b0ab-39fec1295a40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:16.290469 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:16.291491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:16.292338 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:16.299522 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 424/831] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:16.299522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934ee80-85d0-4b96-b0ab-39fec1295a40 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:16.300333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a51f8-be90-4713-98f0-442e35836615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:16.301196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a51f8-be90-4713-98f0-442e35836615 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:16.301641 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a51f8-be90-4713-98f0-442e35836615 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:16.302423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a51f8-be90-4713-98f0-442e35836615 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:16.303278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a51f8-be90-4713-98f0-442e35836615 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:16.304751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a51f8-be90-4713-98f0-442e35836615 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:16.333788 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 408/832] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jul 5 10:00:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:16.335026 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a51f8-be90-4713-98f0-442e35836615 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:16.466987 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:16.469749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a51f8-be90-4713-98f0-442e35836615 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:16.502644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:16.512336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c045b-1402-44aa-a4fa-67f1ac4d8fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:16.513326 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c045b-1402-44aa-a4fa-67f1ac4d8fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:16.513804 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c045b-1402-44aa-a4fa-67f1ac4d8fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:16.514248 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c045b-1402-44aa-a4fa-67f1ac4d8fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:16.515164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c045b-1402-44aa-a4fa-67f1ac4d8fd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:16.515637 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c045b-1402-44aa-a4fa-67f1ac4d8fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:16.516746 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934ee80-85d0-4b96-b0ab-39fec1295a40 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:16.517542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c045b-1402-44aa-a4fa-67f1ac4d8fd4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:16.528817 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 409/833] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:00:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:16.534891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c045b-1402-44aa-a4fa-67f1ac4d8fd4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:16.575129 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717f7ac-d537-433a-873a-ddd743adcf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:16.575397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717f7ac-d537-433a-873a-ddd743adcf12 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:16.575541 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717f7ac-d537-433a-873a-ddd743adcf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:16.575819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717f7ac-d537-433a-873a-ddd743adcf12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:16.585023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717f7ac-d537-433a-873a-ddd743adcf12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:16.586533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717f7ac-d537-433a-873a-ddd743adcf12 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-e717f7ac-d537-433a-873a-ddd743adcf12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:16.601886 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:16.603002 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:16.603900 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:16.603900 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:16.603900 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:16.603900 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:16.603900 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:16.603900 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:16.603900 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:16.603900 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:16.606821 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 425/834] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:16.609247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717f7ac-d537-433a-873a-ddd743adcf12 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:16.658079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c045b-1402-44aa-a4fa-67f1ac4d8fd4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:16.698206 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca202a7b-7512-48cd-a5ae-4816f47654fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:16.699480 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca202a7b-7512-48cd-a5ae-4816f47654fa None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:16.699987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca202a7b-7512-48cd-a5ae-4816f47654fa None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:16.700433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca202a7b-7512-48cd-a5ae-4816f47654fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:16.717372 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca202a7b-7512-48cd-a5ae-4816f47654fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:16.719071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca202a7b-7512-48cd-a5ae-4816f47654fa None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:16.735728 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717f7ac-d537-433a-873a-ddd743adcf12 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ca202a7b-7512-48cd-a5ae-4816f47654fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:16.738571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:16.739453 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:16.740389 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:16.740389 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:16.740389 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:16.740389 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:16.742912 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 410/835] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:16.748659 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca202a7b-7512-48cd-a5ae-4816f47654fa None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:16.782877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b59f78-7c4a-427c-8b19-9ad85c022613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:16.783902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b59f78-7c4a-427c-8b19-9ad85c022613 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:16.784374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b59f78-7c4a-427c-8b19-9ad85c022613 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:16.785405 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b59f78-7c4a-427c-8b19-9ad85c022613 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:16.787388 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b59f78-7c4a-427c-8b19-9ad85c022613 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:16.788190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:16.788427 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b59f78-7c4a-427c-8b19-9ad85c022613 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:16.790851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b59f78-7c4a-427c-8b19-9ad85c022613 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:16.806405 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 426/836] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:16.816422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b59f78-7c4a-427c-8b19-9ad85c022613 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:16.834580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:16.875516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca202a7b-7512-48cd-a5ae-4816f47654fa None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:16.910966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee02de-a5ef-4c65-a13c-c1198c01239b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:16.911743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee02de-a5ef-4c65-a13c-c1198c01239b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:16.912111 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee02de-a5ef-4c65-a13c-c1198c01239b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:16.912505 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee02de-a5ef-4c65-a13c-c1198c01239b None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:16.913300 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee02de-a5ef-4c65-a13c-c1198c01239b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:16.913667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee02de-a5ef-4c65-a13c-c1198c01239b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:16.915234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee02de-a5ef-4c65-a13c-c1198c01239b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:16.924194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b59f78-7c4a-427c-8b19-9ad85c022613 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:16.928883 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 411/837] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:16.934347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee02de-a5ef-4c65-a13c-c1198c01239b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:16.959239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364dfe7b-4534-42ec-8874-6fcc5a0f97cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:16.959907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dfe7b-4534-42ec-8874-6fcc5a0f97cc None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:16.960192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dfe7b-4534-42ec-8874-6fcc5a0f97cc None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:16.960466 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dfe7b-4534-42ec-8874-6fcc5a0f97cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:16.961298 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dfe7b-4534-42ec-8874-6fcc5a0f97cc None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=fee6ee8f99c34aa281bf02ecd3f23216) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:16.961702 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dfe7b-4534-42ec-8874-6fcc5a0f97cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:16.963279 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dfe7b-4534-42ec-8874-6fcc5a0f97cc None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:16.969904 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 427/838] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:16 2022] GET /identity/v3/role_assignments?user.id=fee6ee8f99c34aa281bf02ecd3f23216&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=c184a177010144429f495f3290d28221 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:17.035352 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:17.046407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee02de-a5ef-4c65-a13c-c1198c01239b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:17.084400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cefaa-951f-4176-bd39-b412adc30b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:17.085219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cefaa-951f-4176-bd39-b412adc30b9b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.085502 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cefaa-951f-4176-bd39-b412adc30b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.085776 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cefaa-951f-4176-bd39-b412adc30b9b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:17.094207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cefaa-951f-4176-bd39-b412adc30b9b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:17.095712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cefaa-951f-4176-bd39-b412adc30b9b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:17.101163 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e74df4-db29-490c-8e53-1ba48655ce1f None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.101476 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e74df4-db29-490c-8e53-1ba48655ce1f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.101726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e74df4-db29-490c-8e53-1ba48655ce1f None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:17.103480 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 428/839] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:17.109287 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff8be8-2410-46a5-9783-7cf6d4c4c206 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.109574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff8be8-2410-46a5-9783-7cf6d4c4c206 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.109858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff8be8-2410-46a5-9783-7cf6d4c4c206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-aa4cefaa-951f-4176-bd39-b412adc30b9b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:17.113605 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:17.115953 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:17.118418 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:17.118418 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:17.118418 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:17.118418 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:17.118418 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:17.118418 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:17.118418 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:17.118418 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:17.118418 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 10:00:17.118418 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:17.118418 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 412/840] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:17.122370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cefaa-951f-4176-bd39-b412adc30b9b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:17.129643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3fff8be8-2410-46a5-9783-7cf6d4c4c206 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:17.131108 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3fff8be8-2410-46a5-9783-7cf6d4c4c206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:17.217947 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 429/841] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:17.243950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cefaa-951f-4176-bd39-b412adc30b9b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:17.245265 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff8be8-2410-46a5-9783-7cf6d4c4c206 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:17.247894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:17.286849 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69821716-4cc9-4e93-b2d9-6811d6ed2c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:17.287459 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69821716-4cc9-4e93-b2d9-6811d6ed2c67 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.287746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69821716-4cc9-4e93-b2d9-6811d6ed2c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.288024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69821716-4cc9-4e93-b2d9-6811d6ed2c67 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:17.288877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69821716-4cc9-4e93-b2d9-6811d6ed2c67 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:17.289191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69821716-4cc9-4e93-b2d9-6811d6ed2c67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:17.291249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69821716-4cc9-4e93-b2d9-6811d6ed2c67 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:17.314261 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 413/842] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jul 5 10:00:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:17.321055 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69821716-4cc9-4e93-b2d9-6811d6ed2c67 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:17.336986 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:17.374467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff8be8-2410-46a5-9783-7cf6d4c4c206 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:17.422817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3e3d8-4308-4c93-b746-c712b5aac91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:17.424324 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3e3d8-4308-4c93-b746-c712b5aac91d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.424799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3e3d8-4308-4c93-b746-c712b5aac91d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.425551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3e3d8-4308-4c93-b746-c712b5aac91d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:17.426906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3e3d8-4308-4c93-b746-c712b5aac91d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:17.428297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3e3d8-4308-4c93-b746-c712b5aac91d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:17.439173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69821716-4cc9-4e93-b2d9-6811d6ed2c67 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-c8f3e3d8-4308-4c93-b746-c712b5aac91d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:17.441448 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:17.442491 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:17.443577 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 430/843] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 10:00:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:17.448770 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3e3d8-4308-4c93-b746-c712b5aac91d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:17.490453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8bb7a-4e0e-472e-8559-bdde32f93722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:17.492182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8bb7a-4e0e-472e-8559-bdde32f93722 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.493375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8bb7a-4e0e-472e-8559-bdde32f93722 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.494008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8bb7a-4e0e-472e-8559-bdde32f93722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:17.520309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:17.559452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3e3d8-4308-4c93-b746-c712b5aac91d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:17.581563 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8bb7a-4e0e-472e-8559-bdde32f93722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:17.583852 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8bb7a-4e0e-472e-8559-bdde32f93722 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:17.593520 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813d7f8-e567-4bf7-9272-17dd8301acf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:17.594342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813d7f8-e567-4bf7-9272-17dd8301acf6 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.594699 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813d7f8-e567-4bf7-9272-17dd8301acf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.594833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:17.595104 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813d7f8-e567-4bf7-9272-17dd8301acf6 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:17.595881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813d7f8-e567-4bf7-9272-17dd8301acf6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:17.596263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813d7f8-e567-4bf7-9272-17dd8301acf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:17.597790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813d7f8-e567-4bf7-9272-17dd8301acf6 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:17.603081 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 431/844] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 10:00:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:17.604437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823830-863a-496a-80e5-12ccdddff616 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.604838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823830-863a-496a-80e5-12ccdddff616 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.605170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823830-863a-496a-80e5-12ccdddff616 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:17.610267 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 432/845] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:17.610267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc823830-863a-496a-80e5-12ccdddff616 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:17.693890 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 414/846] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:00:17.695557 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b1d89-00a4-4ea3-ac93-ed881c6f266b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.695916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b1d89-00a4-4ea3-ac93-ed881c6f266b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.696267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b1d89-00a4-4ea3-ac93-ed881c6f266b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:17.712728 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cf2b1d89-00a4-4ea3-ac93-ed881c6f266b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:17.713933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cf2b1d89-00a4-4ea3-ac93-ed881c6f266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:17.720218 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc823830-863a-496a-80e5-12ccdddff616 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:17.749703 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:17.757011 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c151ff-9b26-45f5-8c88-a2aa123e7ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:17.757422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c151ff-9b26-45f5-8c88-a2aa123e7ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.757715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c151ff-9b26-45f5-8c88-a2aa123e7ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.758077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c151ff-9b26-45f5-8c88-a2aa123e7ad6 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:17.758887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c151ff-9b26-45f5-8c88-a2aa123e7ad6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:17.759208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c151ff-9b26-45f5-8c88-a2aa123e7ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:17.760482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c151ff-9b26-45f5-8c88-a2aa123e7ad6 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:17.769633 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 433/847] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 10:00:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:17.770531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c151ff-9b26-45f5-8c88-a2aa123e7ad6 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:17.810720 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 415/848] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:17.826328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b1d89-00a4-4ea3-ac93-ed881c6f266b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:17.875646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:17.902298 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c151ff-9b26-45f5-8c88-a2aa123e7ad6 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:17.939285 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f9f72-eac0-444b-a3e8-8d624341db4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:17.940258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f9f72-eac0-444b-a3e8-8d624341db4e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.941592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f9f72-eac0-444b-a3e8-8d624341db4e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.941592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f9f72-eac0-444b-a3e8-8d624341db4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:17.950815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b1d89-00a4-4ea3-ac93-ed881c6f266b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:17.954336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f9f72-eac0-444b-a3e8-8d624341db4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:17.955804 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f9f72-eac0-444b-a3e8-8d624341db4e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-d05f9f72-eac0-444b-a3e8-8d624341db4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:17.975177 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:17.976342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:17.977274 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:17.977274 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:17.977274 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:17.977274 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:17.977274 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 434/849] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:17.983579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f9f72-eac0-444b-a3e8-8d624341db4e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:17.996227 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb0f95-a106-4100-b692-fbe6eafb95ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:17.997052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb0f95-a106-4100-b692-fbe6eafb95ca None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:17.997330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb0f95-a106-4100-b692-fbe6eafb95ca None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:17.997608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb0f95-a106-4100-b692-fbe6eafb95ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:18.011148 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb0f95-a106-4100-b692-fbe6eafb95ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5cfb0f95-a106-4100-b692-fbe6eafb95ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:18.022398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:18.023350 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:18.024182 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 416/850] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 10:00:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:18.030341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb0f95-a106-4100-b692-fbe6eafb95ca None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:18.044812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:18.115396 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f9f72-eac0-444b-a3e8-8d624341db4e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:18.147873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:18.154211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb0f95-a106-4100-b692-fbe6eafb95ca None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:18.156353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3c925-9578-49fc-9c30-9647bd8fb48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:18.157125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c925-9578-49fc-9c30-9647bd8fb48f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:18.158159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c925-9578-49fc-9c30-9647bd8fb48f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:18.158159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c925-9578-49fc-9c30-9647bd8fb48f None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:18.158706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3c925-9578-49fc-9c30-9647bd8fb48f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:18.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3c925-9578-49fc-9c30-9647bd8fb48f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:18.160738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3c925-9578-49fc-9c30-9647bd8fb48f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:18.176693 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 435/851] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:18.182062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3c925-9578-49fc-9c30-9647bd8fb48f None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:18.197644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc430540-45c5-43f6-ac45-b937511fc59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:18.198605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc430540-45c5-43f6-ac45-b937511fc59e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:18.199139 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc430540-45c5-43f6-ac45-b937511fc59e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:18.199556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc430540-45c5-43f6-ac45-b937511fc59e None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:18.200463 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc430540-45c5-43f6-ac45-b937511fc59e None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:18.200991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc430540-45c5-43f6-ac45-b937511fc59e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:18.202544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc430540-45c5-43f6-ac45-b937511fc59e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:18.216863 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 417/852] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 10:00:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:18.217970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc430540-45c5-43f6-ac45-b937511fc59e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:18.311706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3c925-9578-49fc-9c30-9647bd8fb48f None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:18.321758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:18.336520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc430540-45c5-43f6-ac45-b937511fc59e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:18.352321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b83c8-eefd-451d-a271-13b70ae344bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:18.353630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b83c8-eefd-451d-a271-13b70ae344bd None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:18.354332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b83c8-eefd-451d-a271-13b70ae344bd None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:18.354752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b83c8-eefd-451d-a271-13b70ae344bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:18.364180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b83c8-eefd-451d-a271-13b70ae344bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:18.366658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b83c8-eefd-451d-a271-13b70ae344bd None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:18.373150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4cf2b1-0f01-4d8a-be6e-a7d8d9e1e481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:18.374080 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cf2b1-0f01-4d8a-be6e-a7d8d9e1e481 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:18.374498 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cf2b1-0f01-4d8a-be6e-a7d8d9e1e481 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:18.374872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cf2b1-0f01-4d8a-be6e-a7d8d9e1e481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-454b83c8-eefd-451d-a271-13b70ae344bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:18.384520 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:18.385576 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:18.386735 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:18.386735 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:18.386735 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:18.386735 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:18.386735 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:18.386735 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:18.386735 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:18.386735 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:18.388591 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4cf2b1-0f01-4d8a-be6e-a7d8d9e1e481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:18.389357 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 436/853] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:18.394303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4cf2b1-0f01-4d8a-be6e-a7d8d9e1e481 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:18.395807 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b83c8-eefd-451d-a271-13b70ae344bd None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-2b4cf2b1-0f01-4d8a-be6e-a7d8d9e1e481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:18.413571 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:18.414718 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:18.415819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:18.415819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:18.415819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:18.415819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:18.421465 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 418/854] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:18.423314 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4cf2b1-0f01-4d8a-be6e-a7d8d9e1e481 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:18.439015 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:18.527164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b83c8-eefd-451d-a271-13b70ae344bd None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:18.530792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4cf2b1-0f01-4d8a-be6e-a7d8d9e1e481 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:18.569638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e432b-890e-41b8-aa8f-228cafbc06ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:18.570687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e432b-890e-41b8-aa8f-228cafbc06ed None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:18.570687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e432b-890e-41b8-aa8f-228cafbc06ed None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:18.570886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e432b-890e-41b8-aa8f-228cafbc06ed None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:18.572070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e432b-890e-41b8-aa8f-228cafbc06ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:18.572070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e432b-890e-41b8-aa8f-228cafbc06ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:18.573306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e432b-890e-41b8-aa8f-228cafbc06ed None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:18.574544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7b83b-e3e0-4dae-a216-070ad87e30bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:18.575519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7b83b-e3e0-4dae-a216-070ad87e30bf None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:18.575991 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7b83b-e3e0-4dae-a216-070ad87e30bf None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:18.576450 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7b83b-e3e0-4dae-a216-070ad87e30bf None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:18.577706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7b83b-e3e0-4dae-a216-070ad87e30bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:18.578636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7b83b-e3e0-4dae-a216-070ad87e30bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:18.580097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7b83b-e3e0-4dae-a216-070ad87e30bf None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:18.592628 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 419/855] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:18.595579 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 437/856] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:18.597873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e432b-890e-41b8-aa8f-228cafbc06ed None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:18.603543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7b83b-e3e0-4dae-a216-070ad87e30bf None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:18.637345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:18.689730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:18.729818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e432b-890e-41b8-aa8f-228cafbc06ed None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:18.731519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7b83b-e3e0-4dae-a216-070ad87e30bf None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:18.766581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea61fe-d002-49a7-8735-6a9d6757de95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:18.767433 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea61fe-d002-49a7-8735-6a9d6757de95 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:18.768401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea61fe-d002-49a7-8735-6a9d6757de95 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:18.768401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea61fe-d002-49a7-8735-6a9d6757de95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:18.769985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea61fe-d002-49a7-8735-6a9d6757de95 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=dbfded29b70140a387f7b8c07c296861) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:18.770343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea61fe-d002-49a7-8735-6a9d6757de95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:18.771841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea61fe-d002-49a7-8735-6a9d6757de95 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:18.772126 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b7a52-24e4-4376-b88b-3c2b434dc28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:18.773164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b7a52-24e4-4376-b88b-3c2b434dc28e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:18.773743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b7a52-24e4-4376-b88b-3c2b434dc28e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:18.774209 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b7a52-24e4-4376-b88b-3c2b434dc28e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:18.782754 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b7a52-24e4-4376-b88b-3c2b434dc28e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:18.784739 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b7a52-24e4-4376-b88b-3c2b434dc28e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:18.786638 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 438/857] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:18 2022] GET /identity/v3/role_assignments?user.id=dbfded29b70140a387f7b8c07c296861&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=c184a177010144429f495f3290d28221 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-606b7a52-24e4-4376-b88b-3c2b434dc28e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:18.802296 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:18.803331 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:18.804252 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:18.804252 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:18.804252 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:18.804252 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:18.804252 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:18.804252 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:18.804252 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:18.804252 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:18.804252 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:18.804252 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:18.807189 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 420/858] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jul 5 10:00:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:18.812891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea61fe-d002-49a7-8735-6a9d6757de95 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:18.913339 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:18.923461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea61fe-d002-49a7-8735-6a9d6757de95 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:18.960346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f864dbf-bd4e-4557-9bfd-0660b043c3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:18.961061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f864dbf-bd4e-4557-9bfd-0660b043c3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:18.961346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f864dbf-bd4e-4557-9bfd-0660b043c3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:18.961622 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f864dbf-bd4e-4557-9bfd-0660b043c3d6 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:18.962531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f864dbf-bd4e-4557-9bfd-0660b043c3d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:18.962844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f864dbf-bd4e-4557-9bfd-0660b043c3d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:18.964349 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f864dbf-bd4e-4557-9bfd-0660b043c3d6 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:18.984502 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 439/859] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jul 5 10:00:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:18.990252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b7a52-24e4-4376-b88b-3c2b434dc28e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:19.093875 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b7a52-24e4-4376-b88b-3c2b434dc28e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:19.117825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ea910-b519-4baf-a59c-38678a5e9b55 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:19.118183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ea910-b519-4baf-a59c-38678a5e9b55 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:19.118415 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ea910-b519-4baf-a59c-38678a5e9b55 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:19.120044 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 440/860] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:19.124865 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597eb9-17c6-4468-a06e-73ffe692ed21 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:19.125148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597eb9-17c6-4468-a06e-73ffe692ed21 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:19.125424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597eb9-17c6-4468-a06e-73ffe692ed21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:19.127146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:19.132648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1177d3-4d72-42a4-bdde-06356efabde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:19.133326 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1177d3-4d72-42a4-bdde-06356efabde3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:19.133617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1177d3-4d72-42a4-bdde-06356efabde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:19.133897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1177d3-4d72-42a4-bdde-06356efabde3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:19.151536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-5b597eb9-17c6-4468-a06e-73ffe692ed21 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:19.153647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5b597eb9-17c6-4468-a06e-73ffe692ed21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:19.223422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1177d3-4d72-42a4-bdde-06356efabde3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:19.224807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1177d3-4d72-42a4-bdde-06356efabde3 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:19.225967 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 441/861] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:19.248942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b597eb9-17c6-4468-a06e-73ffe692ed21 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:19.334566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:19.353136 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 421/862] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 10:00:19.358130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1177d3-4d72-42a4-bdde-06356efabde3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:19.363640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b597eb9-17c6-4468-a06e-73ffe692ed21 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:19.391163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:19.406017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9a5b2-cc27-4382-abd6-e68b7d91ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:19.407196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9a5b2-cc27-4382-abd6-e68b7d91ad48 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:19.407490 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9a5b2-cc27-4382-abd6-e68b7d91ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:19.407765 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9a5b2-cc27-4382-abd6-e68b7d91ad48 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:19.408689 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9a5b2-cc27-4382-abd6-e68b7d91ad48 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:19.410139 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9a5b2-cc27-4382-abd6-e68b7d91ad48 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-35d9a5b2-cc27-4382-abd6-e68b7d91ad48 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:19.425857 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:19.426871 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:19.427752 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 442/863] 10.222.0.16 () {56 vars in 1132 bytes} [Tue Jul 5 10:00:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:19.434556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9a5b2-cc27-4382-abd6-e68b7d91ad48 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:19.477409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1177d3-4d72-42a4-bdde-06356efabde3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:19.516102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b197a-eced-4706-a247-a8f6427b26c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:19.517306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b197a-eced-4706-a247-a8f6427b26c2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:19.517869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b197a-eced-4706-a247-a8f6427b26c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:19.518315 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b197a-eced-4706-a247-a8f6427b26c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:19.530263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b197a-eced-4706-a247-a8f6427b26c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:19.532098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b197a-eced-4706-a247-a8f6427b26c2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:19.541479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9a5b2-cc27-4382-abd6-e68b7d91ad48 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-273b197a-eced-4706-a247-a8f6427b26c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:19.552753 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:19.553886 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:19.555075 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:19.555075 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:19.555075 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:19.555075 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:19.561843 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 422/864] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:19.563038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b197a-eced-4706-a247-a8f6427b26c2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:19.578961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c185b-39f5-46a0-9053-20abd31c99aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:19.579944 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c185b-39f5-46a0-9053-20abd31c99aa None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:19.580092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c185b-39f5-46a0-9053-20abd31c99aa None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:19.580468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c185b-39f5-46a0-9053-20abd31c99aa None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:19.581279 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c185b-39f5-46a0-9053-20abd31c99aa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:19.581712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c185b-39f5-46a0-9053-20abd31c99aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:19.583611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c185b-39f5-46a0-9053-20abd31c99aa None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:19.593229 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 443/865] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jul 5 10:00:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:19.594214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c185b-39f5-46a0-9053-20abd31c99aa None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:19.607616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:19.670709 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:19.691199 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b197a-eced-4706-a247-a8f6427b26c2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:19.726362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c185b-39f5-46a0-9053-20abd31c99aa None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:19.732746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49219bfb-facf-40a0-bd36-cec2ab319157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:19.733399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219bfb-facf-40a0-bd36-cec2ab319157 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:19.733688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219bfb-facf-40a0-bd36-cec2ab319157 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:19.733964 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219bfb-facf-40a0-bd36-cec2ab319157 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:19.734746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49219bfb-facf-40a0-bd36-cec2ab319157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:19.735053 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49219bfb-facf-40a0-bd36-cec2ab319157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:19.736844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49219bfb-facf-40a0-bd36-cec2ab319157 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:19.755705 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 423/866] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:19.757318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49219bfb-facf-40a0-bd36-cec2ab319157 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:19.770875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48947929-dad0-4869-b805-c803ecca170a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:19.771360 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48947929-dad0-4869-b805-c803ecca170a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:19.771645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48947929-dad0-4869-b805-c803ecca170a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:19.771920 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48947929-dad0-4869-b805-c803ecca170a None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:19.772631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48947929-dad0-4869-b805-c803ecca170a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:19.772936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48947929-dad0-4869-b805-c803ecca170a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:19.774625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48947929-dad0-4869-b805-c803ecca170a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:19.786260 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 444/867] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jul 5 10:00:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:19.878090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49219bfb-facf-40a0-bd36-cec2ab319157 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:19.891514 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33439738-2e2c-4b9b-b9bb-af4248ae2ed7 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:19.891836 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33439738-2e2c-4b9b-b9bb-af4248ae2ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:19.892119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33439738-2e2c-4b9b-b9bb-af4248ae2ed7 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:19.893673 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 445/868] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:19.899273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e204c3-b62e-402c-be5e-022341b133a0 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:19.899654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e204c3-b62e-402c-be5e-022341b133a0 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:19.900003 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e204c3-b62e-402c-be5e-022341b133a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:19.918478 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-24e204c3-b62e-402c-be5e-022341b133a0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:19.919777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-24e204c3-b62e-402c-be5e-022341b133a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:19.920609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4e53a-9469-492d-89e7-adbe33db5043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:19.921812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e53a-9469-492d-89e7-adbe33db5043 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:19.922313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e53a-9469-492d-89e7-adbe33db5043 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:19.922770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e53a-9469-492d-89e7-adbe33db5043 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:19.930797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4e53a-9469-492d-89e7-adbe33db5043 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:19.932614 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4e53a-9469-492d-89e7-adbe33db5043 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-57b4e53a-9469-492d-89e7-adbe33db5043 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:19.948220 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:19.949184 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:19.950085 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:19.950085 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:19.950085 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:19.950085 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:19.950085 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:19.950085 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:19.950085 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:19.950085 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:19.952773 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 424/869] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:19.959465 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4e53a-9469-492d-89e7-adbe33db5043 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:19.964479 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:19.997515 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:20.002929 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 446/870] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:20.025951 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e204c3-b62e-402c-be5e-022341b133a0 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:20.085635 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4e53a-9469-492d-89e7-adbe33db5043 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:20.133409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce0eb8-366c-4296-ae41-5ca3da59389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:20.134697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce0eb8-366c-4296-ae41-5ca3da59389b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:20.135219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce0eb8-366c-4296-ae41-5ca3da59389b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:20.135707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce0eb8-366c-4296-ae41-5ca3da59389b None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:20.135879 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e204c3-b62e-402c-be5e-022341b133a0 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:20.136855 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce0eb8-366c-4296-ae41-5ca3da59389b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:20.137601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce0eb8-366c-4296-ae41-5ca3da59389b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:20.140119 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce0eb8-366c-4296-ae41-5ca3da59389b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:20.155928 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 425/871] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:20.162318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce0eb8-366c-4296-ae41-5ca3da59389b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:20.177528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f40ef-8278-4c48-b1ad-d33d6ab8a03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:20.178263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f40ef-8278-4c48-b1ad-d33d6ab8a03c None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:20.178541 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f40ef-8278-4c48-b1ad-d33d6ab8a03c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:20.178818 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f40ef-8278-4c48-b1ad-d33d6ab8a03c None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:20.179569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f40ef-8278-4c48-b1ad-d33d6ab8a03c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:20.180955 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f40ef-8278-4c48-b1ad-d33d6ab8a03c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:20.196388 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 447/872] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 10:00:20 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:20.245919 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:20.289136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce0eb8-366c-4296-ae41-5ca3da59389b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:20.337604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d981eb-25e1-46c1-9dde-a759278a1bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:20.338337 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d981eb-25e1-46c1-9dde-a759278a1bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:20.338651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d981eb-25e1-46c1-9dde-a759278a1bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:20.338936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d981eb-25e1-46c1-9dde-a759278a1bd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:20.339779 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d981eb-25e1-46c1-9dde-a759278a1bd0 None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=84a8d62afe964cb49acc236850adbbf3) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:20.340120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d981eb-25e1-46c1-9dde-a759278a1bd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:20.341637 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d981eb-25e1-46c1-9dde-a759278a1bd0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:20.349006 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 426/873] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:20 2022] GET /identity/v3/role_assignments?user.id=84a8d62afe964cb49acc236850adbbf3&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:20.873306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e4ca-3aab-4383-81a8-553d851ce3b3 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:20.873586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e4ca-3aab-4383-81a8-553d851ce3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:20.873852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87e4ca-3aab-4383-81a8-553d851ce3b3 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:20.875492 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 448/874] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:20.880987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36555fcf-063b-4a5a-9b6c-9242eaffc3df None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:20.881460 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36555fcf-063b-4a5a-9b6c-9242eaffc3df None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:20.881560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36555fcf-063b-4a5a-9b6c-9242eaffc3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:20.898825 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-36555fcf-063b-4a5a-9b6c-9242eaffc3df None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:20.900052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-36555fcf-063b-4a5a-9b6c-9242eaffc3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:20.968164 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 427/875] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:20.990059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87e4ca-3aab-4383-81a8-553d851ce3b3 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:21.048995 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:21.090621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87e4ca-3aab-4383-81a8-553d851ce3b3 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:21.122796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3771e-76c5-45c3-a768-b764f835a2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:21.123608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3771e-76c5-45c3-a768-b764f835a2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:21.123885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3771e-76c5-45c3-a768-b764f835a2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:21.124155 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3771e-76c5-45c3-a768-b764f835a2b0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:21.124800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3771e-76c5-45c3-a768-b764f835a2b0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:21.126160 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3771e-76c5-45c3-a768-b764f835a2b0 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-a5d3771e-76c5-45c3-a768-b764f835a2b0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:21.138278 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:21.139289 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:21.140546 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 449/876] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jul 5 10:00:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:21.145758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36555fcf-063b-4a5a-9b6c-9242eaffc3df None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:21.200746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:21.244816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36555fcf-063b-4a5a-9b6c-9242eaffc3df None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:21.281777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82112742-ed7f-49e9-b15c-6cb3e2518cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:21.282489 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82112742-ed7f-49e9-b15c-6cb3e2518cad None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:21.282771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82112742-ed7f-49e9-b15c-6cb3e2518cad None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:21.283047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82112742-ed7f-49e9-b15c-6cb3e2518cad None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:21.283756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82112742-ed7f-49e9-b15c-6cb3e2518cad None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:21.284063 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82112742-ed7f-49e9-b15c-6cb3e2518cad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:21.285510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82112742-ed7f-49e9-b15c-6cb3e2518cad None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:21.290599 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 428/877] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jul 5 10:00:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:21.295928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3771e-76c5-45c3-a768-b764f835a2b0 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:21.394461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3771e-76c5-45c3-a768-b764f835a2b0 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:21.419162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:21.430478 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c351537-b136-4409-bc38-748a65bbe8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:21.431161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c351537-b136-4409-bc38-748a65bbe8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:21.431450 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c351537-b136-4409-bc38-748a65bbe8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:21.431762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c351537-b136-4409-bc38-748a65bbe8d5 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:21.432481 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c351537-b136-4409-bc38-748a65bbe8d5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:21.432790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c351537-b136-4409-bc38-748a65bbe8d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:21.434178 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c351537-b136-4409-bc38-748a65bbe8d5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:21.439435 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 450/878] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jul 5 10:00:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:21.941767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acae221-4d47-410c-87c3-5ff20604729b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:21.942349 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acae221-4d47-410c-87c3-5ff20604729b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:21.942349 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acae221-4d47-410c-87c3-5ff20604729b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:21.943982 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 429/879] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:21.949188 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6128fa-2eb3-4880-9a5d-b94e2b909f9a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:21.949475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6128fa-2eb3-4880-9a5d-b94e2b909f9a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:21.949752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6128fa-2eb3-4880-9a5d-b94e2b909f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:21.966798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ad6128fa-2eb3-4880-9a5d-b94e2b909f9a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:21.967968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ad6128fa-2eb3-4880-9a5d-b94e2b909f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:22.033978 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 451/880] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:22.055731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acae221-4d47-410c-87c3-5ff20604729b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:22.164774 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acae221-4d47-410c-87c3-5ff20604729b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:22.183243 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:22.201408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e8443-5c6e-4960-8d75-e0627798b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:22.202493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8443-5c6e-4960-8d75-e0627798b37a None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:22.202981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8443-5c6e-4960-8d75-e0627798b37a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:22.203484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8443-5c6e-4960-8d75-e0627798b37a None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:22.204434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e8443-5c6e-4960-8d75-e0627798b37a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:22.206045 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e8443-5c6e-4960-8d75-e0627798b37a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:22.224088 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 430/881] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 10:00:22 2022] POST /identity/v3/services => generated 269 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:22.344924 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e9efe-c965-4968-87f8-8a8a7cd4ca42 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:22.345618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e9efe-c965-4968-87f8-8a8a7cd4ca42 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:22.346099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e9efe-c965-4968-87f8-8a8a7cd4ca42 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:22.348112 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 452/882] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:22.353930 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75c42f-1ea8-4c78-815d-5893e2238670 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:22.354560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75c42f-1ea8-4c78-815d-5893e2238670 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:22.355035 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75c42f-1ea8-4c78-815d-5893e2238670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:22.373245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dc75c42f-1ea8-4c78-815d-5893e2238670 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:22.374818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dc75c42f-1ea8-4c78-815d-5893e2238670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:22.459387 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 431/883] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:22.478701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e9efe-c965-4968-87f8-8a8a7cd4ca42 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:22.539623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b4df-aec5-47c0-8420-ceb16f60f547 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:22.540277 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b4df-aec5-47c0-8420-ceb16f60f547 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:22.540816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b4df-aec5-47c0-8420-ceb16f60f547 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:22.542487 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 432/884] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:22.548757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c472-519a-4dac-845e-b1ef32405f8d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:22.549051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c472-519a-4dac-845e-b1ef32405f8d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:22.549335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c472-519a-4dac-845e-b1ef32405f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:22.567278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5a85c472-519a-4dac-845e-b1ef32405f8d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:22.568494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5a85c472-519a-4dac-845e-b1ef32405f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:22.600083 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e9efe-c965-4968-87f8-8a8a7cd4ca42 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:22.618773 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:22.637608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe6835-5498-4721-be3c-90c202351b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:22.638516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe6835-5498-4721-be3c-90c202351b81 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:22.638802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe6835-5498-4721-be3c-90c202351b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:22.639081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe6835-5498-4721-be3c-90c202351b81 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:22.639745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe6835-5498-4721-be3c-90c202351b81 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:22.641120 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe6835-5498-4721-be3c-90c202351b81 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-24fe6835-5498-4721-be3c-90c202351b81 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:22.653111 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:22.654109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 10:00:22.654987 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:22.658295 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 453/885] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 10:00:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:22.660423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe6835-5498-4721-be3c-90c202351b81 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:22.661918 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 433/886] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:22.685785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85c472-519a-4dac-845e-b1ef32405f8d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:22.722519 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:22.790268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe6835-5498-4721-be3c-90c202351b81 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:22.799575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85c472-519a-4dac-845e-b1ef32405f8d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:22.823722 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ec377-38f6-481a-9126-ddad73f02f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:22.824391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec377-38f6-481a-9126-ddad73f02f7d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:22.824673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec377-38f6-481a-9126-ddad73f02f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:22.824948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec377-38f6-481a-9126-ddad73f02f7d None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:22.825654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ec377-38f6-481a-9126-ddad73f02f7d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:22.825959 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ec377-38f6-481a-9126-ddad73f02f7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:22.827368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ec377-38f6-481a-9126-ddad73f02f7d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:22.832799 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 454/887] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 10:00:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:22.838146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ec377-38f6-481a-9126-ddad73f02f7d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:22.850814 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb104277-4f55-424e-8188-c50e4979a985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:22.852002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104277-4f55-424e-8188-c50e4979a985 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:22.852418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104277-4f55-424e-8188-c50e4979a985 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:22.852807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104277-4f55-424e-8188-c50e4979a985 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:22.867550 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb104277-4f55-424e-8188-c50e4979a985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-cb104277-4f55-424e-8188-c50e4979a985 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:22.877688 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:22.878804 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:22.879635 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:22.882335 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 434/888] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jul 5 10:00:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:22.887113 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb104277-4f55-424e-8188-c50e4979a985 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:22.891777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:22.950023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ec377-38f6-481a-9126-ddad73f02f7d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:22.994017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb5c09-690c-483e-8ded-94e56cad7e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:22.995576 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb5c09-690c-483e-8ded-94e56cad7e8b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:22.996057 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb5c09-690c-483e-8ded-94e56cad7e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:22.996757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb5c09-690c-483e-8ded-94e56cad7e8b None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:22.997936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb5c09-690c-483e-8ded-94e56cad7e8b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:22.998849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb5c09-690c-483e-8ded-94e56cad7e8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:23.000875 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:23.002610 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb5c09-690c-483e-8ded-94e56cad7e8b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:23.008902 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb104277-4f55-424e-8188-c50e4979a985 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:23.012037 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 455/889] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 10:00:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:23.016937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb5c09-690c-483e-8ded-94e56cad7e8b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:23.052307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce21ede-812c-4ab3-b681-f3c1f4099fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:23.052990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce21ede-812c-4ab3-b681-f3c1f4099fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:23.053274 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce21ede-812c-4ab3-b681-f3c1f4099fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:23.053925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce21ede-812c-4ab3-b681-f3c1f4099fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:23.054972 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce21ede-812c-4ab3-b681-f3c1f4099fc2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:23.055293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce21ede-812c-4ab3-b681-f3c1f4099fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:23.056777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce21ede-812c-4ab3-b681-f3c1f4099fc2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:23.072263 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 435/890] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jul 5 10:00:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:23.074088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce21ede-812c-4ab3-b681-f3c1f4099fc2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:23.135579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb5c09-690c-483e-8ded-94e56cad7e8b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:23.147711 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:23.174778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd983f1-d064-4ab0-a6e7-aee574839df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:23.174964 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce21ede-812c-4ab3-b681-f3c1f4099fc2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:23.175952 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd983f1-d064-4ab0-a6e7-aee574839df5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:23.176557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd983f1-d064-4ab0-a6e7-aee574839df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:23.177134 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd983f1-d064-4ab0-a6e7-aee574839df5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:23.178123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd983f1-d064-4ab0-a6e7-aee574839df5 None admin] RBAC: Adding query filter params (interface=public, service_id=a71b063d10cf46e381281034fc795334, region_id=RegionOne) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:23.178670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd983f1-d064-4ab0-a6e7-aee574839df5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:23.180239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd983f1-d064-4ab0-a6e7-aee574839df5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:23.190804 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 456/891] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jul 5 10:00:23 2022] GET /identity/v3/endpoints?service_id=a71b063d10cf46e381281034fc795334&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:23.192376 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd983f1-d064-4ab0-a6e7-aee574839df5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:23.208991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d28fc-7684-4f63-a1f0-c17cb4ce7b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:23.209841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d28fc-7684-4f63-a1f0-c17cb4ce7b8f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:23.210075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d28fc-7684-4f63-a1f0-c17cb4ce7b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:23.210392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d28fc-7684-4f63-a1f0-c17cb4ce7b8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:23.221790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d28fc-7684-4f63-a1f0-c17cb4ce7b8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:23.223161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d28fc-7684-4f63-a1f0-c17cb4ce7b8f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-7b7d28fc-7684-4f63-a1f0-c17cb4ce7b8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:23.240939 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:23.241892 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:23.242818 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:23.242818 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:23.242818 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:23.242818 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:23.246765 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 436/892] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:23.247596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d28fc-7684-4f63-a1f0-c17cb4ce7b8f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:23.258850 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:23.307965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd983f1-d064-4ab0-a6e7-aee574839df5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:23.345314 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134ae7d-f9ba-436e-be3a-11750007181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:23.346477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ae7d-f9ba-436e-be3a-11750007181d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:23.347033 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ae7d-f9ba-436e-be3a-11750007181d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:23.347523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ae7d-f9ba-436e-be3a-11750007181d None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:23.348462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134ae7d-f9ba-436e-be3a-11750007181d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:23.349282 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d28fc-7684-4f63-a1f0-c17cb4ce7b8f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:23.350313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134ae7d-f9ba-436e-be3a-11750007181d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:23.366557 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 457/893] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jul 5 10:00:23 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:00:23.389292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974ef03-f917-4f7a-a61e-817308cb207e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:23.389893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974ef03-f917-4f7a-a61e-817308cb207e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:23.390234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974ef03-f917-4f7a-a61e-817308cb207e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:23.390603 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974ef03-f917-4f7a-a61e-817308cb207e None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:23.391346 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974ef03-f917-4f7a-a61e-817308cb207e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:23.391651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974ef03-f917-4f7a-a61e-817308cb207e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:23.393039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974ef03-f917-4f7a-a61e-817308cb207e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:23.406903 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 437/894] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:23.416151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134ae7d-f9ba-436e-be3a-11750007181d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:23.450526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ec1fe-43a8-4e0c-a180-111fafec66db None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:23.450860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ec1fe-43a8-4e0c-a180-111fafec66db None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:23.451078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ec1fe-43a8-4e0c-a180-111fafec66db None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:23.452784 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 438/895] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:23.460841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:23.461666 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e8a73-9c3b-4d82-8aa1-bb828c641534 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:23.461959 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e8a73-9c3b-4d82-8aa1-bb828c641534 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:23.462251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e8a73-9c3b-4d82-8aa1-bb828c641534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:23.481682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-794e8a73-9c3b-4d82-8aa1-bb828c641534 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:23.483056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-794e8a73-9c3b-4d82-8aa1-bb828c641534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:23.526697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134ae7d-f9ba-436e-be3a-11750007181d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:23.565434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:23.571020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15734f3b-fa1a-44f2-970c-0e7ef3a38b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:23.571945 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15734f3b-fa1a-44f2-970c-0e7ef3a38b28 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:23.572315 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15734f3b-fa1a-44f2-970c-0e7ef3a38b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:23.572844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15734f3b-fa1a-44f2-970c-0e7ef3a38b28 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:23.582255 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 439/896] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:23.588883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15734f3b-fa1a-44f2-970c-0e7ef3a38b28 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:23.590641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15734f3b-fa1a-44f2-970c-0e7ef3a38b28 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:23.598178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e8a73-9c3b-4d82-8aa1-bb828c641534 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-15734f3b-fa1a-44f2-970c-0e7ef3a38b28 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:23.606246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:23.607242 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:23.608220 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:23.608220 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:23.608220 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:23.608220 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:23.608220 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:23.608220 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:23.608220 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:23.608220 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:23.608220 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:23.608220 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:23.611038 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 458/897] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jul 5 10:00:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:23.618580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15734f3b-fa1a-44f2-970c-0e7ef3a38b28 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:23.724441 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:23.728909 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15734f3b-fa1a-44f2-970c-0e7ef3a38b28 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:23.730554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e8a73-9c3b-4d82-8aa1-bb828c641534 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:23.770596 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0a9a2-5d2b-4710-900a-8b3db4973643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:23.771227 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0a9a2-5d2b-4710-900a-8b3db4973643 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:23.771606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0a9a2-5d2b-4710-900a-8b3db4973643 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:23.771976 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0a9a2-5d2b-4710-900a-8b3db4973643 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:23.773102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0a9a2-5d2b-4710-900a-8b3db4973643 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:23.773524 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0a9a2-5d2b-4710-900a-8b3db4973643 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:23.775433 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0a9a2-5d2b-4710-900a-8b3db4973643 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:23.776420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe9a4d-b9df-4e49-8cbf-9193f6b6c18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:23.777509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe9a4d-b9df-4e49-8cbf-9193f6b6c18e None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:23.777943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe9a4d-b9df-4e49-8cbf-9193f6b6c18e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:23.779244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe9a4d-b9df-4e49-8cbf-9193f6b6c18e None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:23.780258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe9a4d-b9df-4e49-8cbf-9193f6b6c18e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:23.781783 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe9a4d-b9df-4e49-8cbf-9193f6b6c18e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:23.802260 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 459/898] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jul 5 10:00:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:23.806311 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 440/899] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 10:00:23 2022] POST /identity/v3/services => generated 241 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:00:23.811676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0a9a2-5d2b-4710-900a-8b3db4973643 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:23.927527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0a9a2-5d2b-4710-900a-8b3db4973643 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:23.971031 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42837722-7700-4c0f-a1d2-2b3332409747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:23.972101 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42837722-7700-4c0f-a1d2-2b3332409747 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:23.972384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42837722-7700-4c0f-a1d2-2b3332409747 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:23.972656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42837722-7700-4c0f-a1d2-2b3332409747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:23.996715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:24.045525 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42837722-7700-4c0f-a1d2-2b3332409747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:24.047021 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42837722-7700-4c0f-a1d2-2b3332409747 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:24.146763 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 460/900] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:23 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:00:24.152524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe9a4d-b9df-4e49-8cbf-9193f6b6c18e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:24.213572 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:24.259067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe9a4d-b9df-4e49-8cbf-9193f6b6c18e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:24.293435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f8c4a-c626-47cc-a3cf-f2a7295f15d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:24.293751 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7db51-bb1d-4db4-9696-2aed24f5e990 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:24.293994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7db51-bb1d-4db4-9696-2aed24f5e990 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:24.294265 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8c4a-c626-47cc-a3cf-f2a7295f15d0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:24.294388 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7db51-bb1d-4db4-9696-2aed24f5e990 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:24.294545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8c4a-c626-47cc-a3cf-f2a7295f15d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:24.294825 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8c4a-c626-47cc-a3cf-f2a7295f15d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:24.295802 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 461/901] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:24.300588 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382ea45-7a65-4c83-a931-24975e317d54 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:24.300874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382ea45-7a65-4c83-a931-24975e317d54 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:24.301153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382ea45-7a65-4c83-a931-24975e317d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:24.307961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f8c4a-c626-47cc-a3cf-f2a7295f15d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:24.309188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f8c4a-c626-47cc-a3cf-f2a7295f15d0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:24.318866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3382ea45-7a65-4c83-a931-24975e317d54 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:24.320203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3382ea45-7a65-4c83-a931-24975e317d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5a1f8c4a-c626-47cc-a3cf-f2a7295f15d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:24.326356 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:24.327455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:24.328398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:24.328398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:24.328398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:24.328398 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:24.328398 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 441/902] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:24.333547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f8c4a-c626-47cc-a3cf-f2a7295f15d0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:24.382523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:24.406429 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 462/903] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:24.429119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382ea45-7a65-4c83-a931-24975e317d54 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:24.458482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f8c4a-c626-47cc-a3cf-f2a7295f15d0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:24.474617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:24.502108 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5179ec-3d61-45bb-a7fd-37ca49dad554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:24.503341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5179ec-3d61-45bb-a7fd-37ca49dad554 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:24.503840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5179ec-3d61-45bb-a7fd-37ca49dad554 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:24.504380 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5179ec-3d61-45bb-a7fd-37ca49dad554 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:24.505371 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5179ec-3d61-45bb-a7fd-37ca49dad554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:24.505883 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5179ec-3d61-45bb-a7fd-37ca49dad554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:24.507565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5179ec-3d61-45bb-a7fd-37ca49dad554 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:24.526708 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 442/904] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:24.527918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5179ec-3d61-45bb-a7fd-37ca49dad554 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:24.543430 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382ea45-7a65-4c83-a931-24975e317d54 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:24.577623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741875d-93f7-4b93-a252-1e7c937f7f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:24.578489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741875d-93f7-4b93-a252-1e7c937f7f85 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:24.578781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741875d-93f7-4b93-a252-1e7c937f7f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:24.579020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741875d-93f7-4b93-a252-1e7c937f7f85 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:24.579672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741875d-93f7-4b93-a252-1e7c937f7f85 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:24.581043 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741875d-93f7-4b93-a252-1e7c937f7f85 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-8741875d-93f7-4b93-a252-1e7c937f7f85 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:24.593847 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:24.594851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:24.595701 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 463/905] 10.222.0.16 () {56 vars in 1132 bytes} [Tue Jul 5 10:00:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:24.601205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741875d-93f7-4b93-a252-1e7c937f7f85 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:24.643036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5179ec-3d61-45bb-a7fd-37ca49dad554 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:24.648064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:24.680310 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a21541-e075-408e-87c4-4ab7b730da69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:24.681540 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a21541-e075-408e-87c4-4ab7b730da69 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:24.682091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a21541-e075-408e-87c4-4ab7b730da69 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:24.682603 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a21541-e075-408e-87c4-4ab7b730da69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:24.690038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a21541-e075-408e-87c4-4ab7b730da69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:24.691810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a21541-e075-408e-87c4-4ab7b730da69 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:24.700838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741875d-93f7-4b93-a252-1e7c937f7f85 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f9a21541-e075-408e-87c4-4ab7b730da69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:24.706478 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:24.707697 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:24.708602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:24.708602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:24.708602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:24.708602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:24.708602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:24.708602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:24.708602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:24.708602 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:24.714582 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 443/906] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:24.717056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a21541-e075-408e-87c4-4ab7b730da69 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:24.734452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5b01a-09a8-4387-bf54-87c5568925c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:24.735122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5b01a-09a8-4387-bf54-87c5568925c5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:24.735402 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5b01a-09a8-4387-bf54-87c5568925c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:24.735675 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5b01a-09a8-4387-bf54-87c5568925c5 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:24.736374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5b01a-09a8-4387-bf54-87c5568925c5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:24.736676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5b01a-09a8-4387-bf54-87c5568925c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:24.738098 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5b01a-09a8-4387-bf54-87c5568925c5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:24.747514 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 464/907] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jul 5 10:00:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:24.748411 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5b01a-09a8-4387-bf54-87c5568925c5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:24.751531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:24.826899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a21541-e075-408e-87c4-4ab7b730da69 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:24.844969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5b01a-09a8-4387-bf54-87c5568925c5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:24.855489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:24.864134 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee40fb-b469-4df5-9781-e97452ebafd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:24.864895 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee40fb-b469-4df5-9781-e97452ebafd1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:24.865281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee40fb-b469-4df5-9781-e97452ebafd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:24.865657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee40fb-b469-4df5-9781-e97452ebafd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:24.866592 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee40fb-b469-4df5-9781-e97452ebafd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:24.867001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee40fb-b469-4df5-9781-e97452ebafd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:24.868536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee40fb-b469-4df5-9781-e97452ebafd1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:24.882379 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 444/908] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:24.886499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78c6e9-1343-49b1-9c5c-83b4f32b7cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:24.887181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78c6e9-1343-49b1-9c5c-83b4f32b7cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:24.887461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78c6e9-1343-49b1-9c5c-83b4f32b7cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:24.887733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78c6e9-1343-49b1-9c5c-83b4f32b7cb7 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:24.888447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78c6e9-1343-49b1-9c5c-83b4f32b7cb7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:24.888750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78c6e9-1343-49b1-9c5c-83b4f32b7cb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:24.890115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78c6e9-1343-49b1-9c5c-83b4f32b7cb7 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:24.891276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee40fb-b469-4df5-9781-e97452ebafd1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:24.900467 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 465/909] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jul 5 10:00:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:24.901641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78c6e9-1343-49b1-9c5c-83b4f32b7cb7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:24.988609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:25.000759 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee40fb-b469-4df5-9781-e97452ebafd1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:25.000922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78c6e9-1343-49b1-9c5c-83b4f32b7cb7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:25.034733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf14b01-a9c9-42ec-82b2-dbe3a82f1f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:25.035401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf14b01-a9c9-42ec-82b2-dbe3a82f1f12 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:25.035772 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf14b01-a9c9-42ec-82b2-dbe3a82f1f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:25.035948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf14b01-a9c9-42ec-82b2-dbe3a82f1f12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:25.036723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf14b01-a9c9-42ec-82b2-dbe3a82f1f12 None admin] RBAC: Adding query filter params (interface=public, service_id=62dd73b8d248420aafc6e22ef807c6bc, region_id=RegionOne) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:25.037030 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf14b01-a9c9-42ec-82b2-dbe3a82f1f12 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:25.038435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf14b01-a9c9-42ec-82b2-dbe3a82f1f12 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:25.043610 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf6297-afc6-44ff-a98c-3cbf5482507b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:25.044561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf6297-afc6-44ff-a98c-3cbf5482507b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:25.045070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf6297-afc6-44ff-a98c-3cbf5482507b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:25.045610 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf6297-afc6-44ff-a98c-3cbf5482507b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:25.046840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf6297-afc6-44ff-a98c-3cbf5482507b None admin] RBAC: Adding query filter params (role.id=9f8e96f6abc44873a5d208d80eb06f0c, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=84a8d62afe964cb49acc236850adbbf3) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:25.047424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf6297-afc6-44ff-a98c-3cbf5482507b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:25.048302 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 466/910] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jul 5 10:00:24 2022] GET /identity/v3/endpoints?service_id=62dd73b8d248420aafc6e22ef807c6bc&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:25.049273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf14b01-a9c9-42ec-82b2-dbe3a82f1f12 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:25.049763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf6297-afc6-44ff-a98c-3cbf5482507b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:25.066578 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 445/911] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:24 2022] GET /identity/v3/role_assignments?user.id=84a8d62afe964cb49acc236850adbbf3&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=9f8e96f6abc44873a5d208d80eb06f0c => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:25.103919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:25.174755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf14b01-a9c9-42ec-82b2-dbe3a82f1f12 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:25.212456 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c660a9-a5bf-4b5b-b999-cc9384ae210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:25.213145 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c660a9-a5bf-4b5b-b999-cc9384ae210d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:25.213431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c660a9-a5bf-4b5b-b999-cc9384ae210d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:25.213710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c660a9-a5bf-4b5b-b999-cc9384ae210d None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:25.214444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c660a9-a5bf-4b5b-b999-cc9384ae210d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:25.215847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c660a9-a5bf-4b5b-b999-cc9384ae210d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:25.222922 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 467/912] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jul 5 10:00:25 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:00:25.432385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a618b6-16cb-4295-a56b-4afa8ef8588c None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:25.432883 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a618b6-16cb-4295-a56b-4afa8ef8588c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:25.432883 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a618b6-16cb-4295-a56b-4afa8ef8588c None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:25.434536 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 446/913] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:25.439765 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630250a-34d8-45de-ace3-c23fc7229df9 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:25.440056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630250a-34d8-45de-ace3-c23fc7229df9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:25.440337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630250a-34d8-45de-ace3-c23fc7229df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:25.459913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-4630250a-34d8-45de-ace3-c23fc7229df9 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:25.461395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4630250a-34d8-45de-ace3-c23fc7229df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:25.542328 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 468/914] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:25.558574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a618b6-16cb-4295-a56b-4afa8ef8588c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:25.663340 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a618b6-16cb-4295-a56b-4afa8ef8588c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:25.698649 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733151da-5c38-4c5b-8d82-c3f2ec3d3b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:25.699386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733151da-5c38-4c5b-8d82-c3f2ec3d3b68 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:25.699671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733151da-5c38-4c5b-8d82-c3f2ec3d3b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:25.699953 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733151da-5c38-4c5b-8d82-c3f2ec3d3b68 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:25.700619 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733151da-5c38-4c5b-8d82-c3f2ec3d3b68 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:25.701998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733151da-5c38-4c5b-8d82-c3f2ec3d3b68 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-733151da-5c38-4c5b-8d82-c3f2ec3d3b68 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:25.714798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:25.715863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:25.717119 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 447/915] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 10:00:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:25.724434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630250a-34d8-45de-ace3-c23fc7229df9 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:25.778585 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:25.828375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630250a-34d8-45de-ace3-c23fc7229df9 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:25.862484 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402972b9-7eca-4575-8974-19ef8e64d659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:25.863156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402972b9-7eca-4575-8974-19ef8e64d659 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:25.863437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402972b9-7eca-4575-8974-19ef8e64d659 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:25.863709 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402972b9-7eca-4575-8974-19ef8e64d659 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:25.864422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402972b9-7eca-4575-8974-19ef8e64d659 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:25.864728 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402972b9-7eca-4575-8974-19ef8e64d659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:25.866107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402972b9-7eca-4575-8974-19ef8e64d659 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:25.871263 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 469/916] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 10:00:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:25.876902 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733151da-5c38-4c5b-8d82-c3f2ec3d3b68 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:25.896759 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:25.978061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733151da-5c38-4c5b-8d82-c3f2ec3d3b68 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:26.011516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d762d-1d93-4f78-bd4b-d99b3b57be25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:26.012497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d762d-1d93-4f78-bd4b-d99b3b57be25 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:26.012981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d762d-1d93-4f78-bd4b-d99b3b57be25 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:26.013483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d762d-1d93-4f78-bd4b-d99b3b57be25 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:26.014512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d762d-1d93-4f78-bd4b-d99b3b57be25 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:26.015032 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d762d-1d93-4f78-bd4b-d99b3b57be25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:26.016743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d762d-1d93-4f78-bd4b-d99b3b57be25 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:26.017197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67757fd0-a1e3-43cb-9541-6651cffce85f None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:26.017490 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67757fd0-a1e3-43cb-9541-6651cffce85f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:26.017767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67757fd0-a1e3-43cb-9541-6651cffce85f None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:26.019760 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 470/917] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:26.026546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c545ed3-6806-45ad-8e83-a2dc0a4eb821 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:26.026546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c545ed3-6806-45ad-8e83-a2dc0a4eb821 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:26.026546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c545ed3-6806-45ad-8e83-a2dc0a4eb821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:26.026817 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 448/918] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jul 5 10:00:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:26.029527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d762d-1d93-4f78-bd4b-d99b3b57be25 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:26.044553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-1c545ed3-6806-45ad-8e83-a2dc0a4eb821 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:26.046044 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1c545ed3-6806-45ad-8e83-a2dc0a4eb821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:26.125827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:26.138654 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 471/919] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:26.152156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d762d-1d93-4f78-bd4b-d99b3b57be25 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:26.161390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c545ed3-6806-45ad-8e83-a2dc0a4eb821 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:26.192169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:26.199979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed5b64-8928-451a-9a4d-a6d41deadc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:26.201193 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed5b64-8928-451a-9a4d-a6d41deadc15 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:26.201724 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed5b64-8928-451a-9a4d-a6d41deadc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:26.202730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed5b64-8928-451a-9a4d-a6d41deadc15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:26.203693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed5b64-8928-451a-9a4d-a6d41deadc15 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:26.205301 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed5b64-8928-451a-9a4d-a6d41deadc15 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:26.232856 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 449/920] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jul 5 10:00:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:26.286850 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c545ed3-6806-45ad-8e83-a2dc0a4eb821 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:26.338661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbd754-13c4-4323-9daf-1b68112a649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:26.339493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbd754-13c4-4323-9daf-1b68112a649d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:26.339774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbd754-13c4-4323-9daf-1b68112a649d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:26.340047 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbd754-13c4-4323-9daf-1b68112a649d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:26.340710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbd754-13c4-4323-9daf-1b68112a649d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:26.342116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbd754-13c4-4323-9daf-1b68112a649d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-62cbd754-13c4-4323-9daf-1b68112a649d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:26.355150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:26.356217 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:26.357054 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 472/921] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jul 5 10:00:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:26.364159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed5b64-8928-451a-9a4d-a6d41deadc15 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:26.471611 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed5b64-8928-451a-9a4d-a6d41deadc15 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:26.506747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede505-7093-4a10-96eb-6228f451d639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:26.507777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede505-7093-4a10-96eb-6228f451d639 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:26.508280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede505-7093-4a10-96eb-6228f451d639 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:26.508777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede505-7093-4a10-96eb-6228f451d639 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:26.509710 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ede505-7093-4a10-96eb-6228f451d639 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:26.510240 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ede505-7093-4a10-96eb-6228f451d639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:26.511814 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ede505-7093-4a10-96eb-6228f451d639 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:26.518155 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 450/922] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jul 5 10:00:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:26.527306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbd754-13c4-4323-9daf-1b68112a649d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:26.623993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:26.639061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbd754-13c4-4323-9daf-1b68112a649d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:26.671980 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c3b5f-86e8-4887-a571-ab2330f200a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:26.672641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c3b5f-86e8-4887-a571-ab2330f200a7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:26.672923 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c3b5f-86e8-4887-a571-ab2330f200a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:26.673193 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c3b5f-86e8-4887-a571-ab2330f200a7 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:26.673896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c3b5f-86e8-4887-a571-ab2330f200a7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:26.674197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c3b5f-86e8-4887-a571-ab2330f200a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:26.675641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c3b5f-86e8-4887-a571-ab2330f200a7 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:26.680774 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 473/923] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jul 5 10:00:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:26.686681 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede505-7093-4a10-96eb-6228f451d639 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:26.689821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:26.809786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede505-7093-4a10-96eb-6228f451d639 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:26.843588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5485c1-ed00-450d-ac63-b909724323ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:26.844266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5485c1-ed00-450d-ac63-b909724323ce None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:26.844558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5485c1-ed00-450d-ac63-b909724323ce None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:26.844839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5485c1-ed00-450d-ac63-b909724323ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:26.845642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5485c1-ed00-450d-ac63-b909724323ce None admin] RBAC: Adding query filter params (interface=public, service_id=2c12dc0172a34d17bbaebd8a0eadb271, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:26.845956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5485c1-ed00-450d-ac63-b909724323ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:26.847457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5485c1-ed00-450d-ac63-b909724323ce None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:26.853050 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 451/924] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jul 5 10:00:26 2022] GET /identity/v3/endpoints?service_id=2c12dc0172a34d17bbaebd8a0eadb271&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:26.858581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c3b5f-86e8-4887-a571-ab2330f200a7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:26.962003 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c3b5f-86e8-4887-a571-ab2330f200a7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:26.995170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8d157-bc6e-4e94-a81f-fd71a5e4ed80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:26.995841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8d157-bc6e-4e94-a81f-fd71a5e4ed80 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:26.996122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8d157-bc6e-4e94-a81f-fd71a5e4ed80 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:26.996395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8d157-bc6e-4e94-a81f-fd71a5e4ed80 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:26.997084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8d157-bc6e-4e94-a81f-fd71a5e4ed80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:26.998477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8d157-bc6e-4e94-a81f-fd71a5e4ed80 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:27.004994 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 474/925] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jul 5 10:00:26 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:00:27.280629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643207c-07cf-46b3-bfbe-1fbb1b6e184f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:27.281042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643207c-07cf-46b3-bfbe-1fbb1b6e184f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:27.281240 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643207c-07cf-46b3-bfbe-1fbb1b6e184f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:27.282814 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 452/926] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:27.289188 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f431b61-38be-47da-9838-c3449e3db2e5 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:27.289696 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f431b61-38be-47da-9838-c3449e3db2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:27.289943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f431b61-38be-47da-9838-c3449e3db2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:27.291705 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8871358-b607-4731-bda7-d4da007ade9e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:27.292056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8871358-b607-4731-bda7-d4da007ade9e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:27.292418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8871358-b607-4731-bda7-d4da007ade9e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:27.294047 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 453/927] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:27.298825 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb89d-e1fc-4481-a26e-801fef5a6ee1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:27.299110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb89d-e1fc-4481-a26e-801fef5a6ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:27.299393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb89d-e1fc-4481-a26e-801fef5a6ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:27.307786 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-6f431b61-38be-47da-9838-c3449e3db2e5 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:27.308959 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6f431b61-38be-47da-9838-c3449e3db2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:27.315965 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-48ffb89d-e1fc-4481-a26e-801fef5a6ee1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:27.317215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-48ffb89d-e1fc-4481-a26e-801fef5a6ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:27.318704 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:27.380231 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 475/928] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:27.395180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:27.395432 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 454/929] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:27.401983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f431b61-38be-47da-9838-c3449e3db2e5 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:27.420680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffb89d-e1fc-4481-a26e-801fef5a6ee1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:27.512404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f431b61-38be-47da-9838-c3449e3db2e5 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:27.535172 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffb89d-e1fc-4481-a26e-801fef5a6ee1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:27.549873 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252d0aa-2165-4fd5-b767-3a19dc2e52a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:27.550856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252d0aa-2165-4fd5-b767-3a19dc2e52a8 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:27.551231 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252d0aa-2165-4fd5-b767-3a19dc2e52a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:27.551573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252d0aa-2165-4fd5-b767-3a19dc2e52a8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:27.552398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252d0aa-2165-4fd5-b767-3a19dc2e52a8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:27.553949 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252d0aa-2165-4fd5-b767-3a19dc2e52a8 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-5252d0aa-2165-4fd5-b767-3a19dc2e52a8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:27.567075 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:27.568029 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:27.568874 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 476/930] 10.222.0.16 () {56 vars in 1132 bytes} [Tue Jul 5 10:00:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:27.572326 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cc93e-c56e-4cce-9b85-396083a95f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:27.578244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cc93e-c56e-4cce-9b85-396083a95f76 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:27.578244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cc93e-c56e-4cce-9b85-396083a95f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:27.578244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cc93e-c56e-4cce-9b85-396083a95f76 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:27.578768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252d0aa-2165-4fd5-b767-3a19dc2e52a8 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:27.587906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2cc93e-c56e-4cce-9b85-396083a95f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:27.588187 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-db2cc93e-c56e-4cce-9b85-396083a95f76 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:27.599302 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:27.601319 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:27.602422 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 455/931] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jul 5 10:00:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:27.606290 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cc93e-c56e-4cce-9b85-396083a95f76 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:27.675238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:27.695705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252d0aa-2165-4fd5-b767-3a19dc2e52a8 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:27.724493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cc93e-c56e-4cce-9b85-396083a95f76 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:27.731789 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0c453-576e-4a97-ab92-97409c8e5391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:27.732608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0c453-576e-4a97-ab92-97409c8e5391 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:27.732950 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0c453-576e-4a97-ab92-97409c8e5391 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:27.733282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0c453-576e-4a97-ab92-97409c8e5391 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:27.734153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0c453-576e-4a97-ab92-97409c8e5391 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:27.734548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0c453-576e-4a97-ab92-97409c8e5391 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:27.736064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0c453-576e-4a97-ab92-97409c8e5391 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:27.745952 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 477/932] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jul 5 10:00:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:27.747036 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0c453-576e-4a97-ab92-97409c8e5391 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:27.759456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b46a11-5ccd-4983-b8f7-94c9f3a54db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:27.760190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b46a11-5ccd-4983-b8f7-94c9f3a54db1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:27.760484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b46a11-5ccd-4983-b8f7-94c9f3a54db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:27.760761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b46a11-5ccd-4983-b8f7-94c9f3a54db1 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:27.761521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b46a11-5ccd-4983-b8f7-94c9f3a54db1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:27.761826 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b46a11-5ccd-4983-b8f7-94c9f3a54db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:27.763773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b46a11-5ccd-4983-b8f7-94c9f3a54db1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:27.776887 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 456/933] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jul 5 10:00:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:27.777821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b46a11-5ccd-4983-b8f7-94c9f3a54db1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:27.833287 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:27.865460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0c453-576e-4a97-ab92-97409c8e5391 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:27.883679 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b46a11-5ccd-4983-b8f7-94c9f3a54db1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:27.900972 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5fb03-e06a-416a-a76e-61e635ea30a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:27.901446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:27.903065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5fb03-e06a-416a-a76e-61e635ea30a0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:27.903496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5fb03-e06a-416a-a76e-61e635ea30a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:27.904036 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5fb03-e06a-416a-a76e-61e635ea30a0 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:27.904985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5fb03-e06a-416a-a76e-61e635ea30a0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:27.906156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5fb03-e06a-416a-a76e-61e635ea30a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:27.908000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5fb03-e06a-416a-a76e-61e635ea30a0 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:27.918532 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 478/934] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jul 5 10:00:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:27.918927 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bcb6a-4c51-4fb8-9de2-b5eb57300842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:27.919729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bcb6a-4c51-4fb8-9de2-b5eb57300842 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:27.919899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5fb03-e06a-416a-a76e-61e635ea30a0 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:27.920061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bcb6a-4c51-4fb8-9de2-b5eb57300842 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:27.920341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bcb6a-4c51-4fb8-9de2-b5eb57300842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:27.931534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bcb6a-4c51-4fb8-9de2-b5eb57300842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:27.932982 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bcb6a-4c51-4fb8-9de2-b5eb57300842 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-046bcb6a-4c51-4fb8-9de2-b5eb57300842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:27.950279 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:27.951530 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:27.952604 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:27.952604 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:27.952604 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:27.952604 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:27.956169 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 457/935] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:27.957328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bcb6a-4c51-4fb8-9de2-b5eb57300842 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:28.027088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5fb03-e06a-416a-a76e-61e635ea30a0 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:28.056597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bcb6a-4c51-4fb8-9de2-b5eb57300842 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:28.063143 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04d9c9-cd53-4564-aa92-a3d205c6e037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:28.064119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04d9c9-cd53-4564-aa92-a3d205c6e037 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:28.064545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04d9c9-cd53-4564-aa92-a3d205c6e037 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:28.064957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04d9c9-cd53-4564-aa92-a3d205c6e037 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:28.066006 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04d9c9-cd53-4564-aa92-a3d205c6e037 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:28.067656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04d9c9-cd53-4564-aa92-a3d205c6e037 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:28.074488 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:28.094937 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 479/936] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jul 5 10:00:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:28.097090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02c53f-e812-493d-8f77-a418c15c0436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:28.097778 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02c53f-e812-493d-8f77-a418c15c0436 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:28.098059 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02c53f-e812-493d-8f77-a418c15c0436 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:28.098363 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02c53f-e812-493d-8f77-a418c15c0436 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:28.099101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02c53f-e812-493d-8f77-a418c15c0436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:28.099406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02c53f-e812-493d-8f77-a418c15c0436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:28.100810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02c53f-e812-493d-8f77-a418c15c0436 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:28.115027 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 458/937] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:28.120987 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04d9c9-cd53-4564-aa92-a3d205c6e037 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:28.231620 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04d9c9-cd53-4564-aa92-a3d205c6e037 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:28.267901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760120b-f5bf-4412-999d-8c326c287540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:28.269114 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760120b-f5bf-4412-999d-8c326c287540 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:28.269569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760120b-f5bf-4412-999d-8c326c287540 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:28.270043 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760120b-f5bf-4412-999d-8c326c287540 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:28.282122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760120b-f5bf-4412-999d-8c326c287540 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:28.283878 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760120b-f5bf-4412-999d-8c326c287540 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-3760120b-f5bf-4412-999d-8c326c287540 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:28.299054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:28.301120 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:28.302581 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:28.302581 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:28.302581 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:28.302581 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:28.302581 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:28.302581 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:28.302581 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:28.302581 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:28.302581 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:28.302581 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:28.302581 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 480/938] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jul 5 10:00:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:28.308272 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02c53f-e812-493d-8f77-a418c15c0436 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:28.340433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:28.358256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3569fa8-6e33-4d7d-9a2f-f7b59002c854 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:28.358529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3569fa8-6e33-4d7d-9a2f-f7b59002c854 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:28.358754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3569fa8-6e33-4d7d-9a2f-f7b59002c854 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:28.360285 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 481/939] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:28.365411 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619929d5-da08-44cb-9c08-db2ad79db879 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:28.365920 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619929d5-da08-44cb-9c08-db2ad79db879 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:28.366544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619929d5-da08-44cb-9c08-db2ad79db879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:28.387922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-619929d5-da08-44cb-9c08-db2ad79db879 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:28.390077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-619929d5-da08-44cb-9c08-db2ad79db879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:28.429174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02c53f-e812-493d-8f77-a418c15c0436 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:28.446509 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:28.474330 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 482/940] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:28.476337 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c990606-d1ed-444f-940f-a6249e3e39a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:28.477182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c990606-d1ed-444f-940f-a6249e3e39a8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:28.477623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c990606-d1ed-444f-940f-a6249e3e39a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:28.478028 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c990606-d1ed-444f-940f-a6249e3e39a8 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:28.479050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c990606-d1ed-444f-940f-a6249e3e39a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:28.479535 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c990606-d1ed-444f-940f-a6249e3e39a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:28.481135 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c990606-d1ed-444f-940f-a6249e3e39a8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:28.496091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619929d5-da08-44cb-9c08-db2ad79db879 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:28.503508 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 459/941] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jul 5 10:00:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:28.509598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c990606-d1ed-444f-940f-a6249e3e39a8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:28.601117 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:28.616650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619929d5-da08-44cb-9c08-db2ad79db879 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:28.617564 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c990606-d1ed-444f-940f-a6249e3e39a8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:28.656560 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be0147-8505-4dc4-bac5-ba490c19b86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:28.657849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be0147-8505-4dc4-bac5-ba490c19b86a None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:28.658365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be0147-8505-4dc4-bac5-ba490c19b86a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:28.658813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac49d34-e832-4d32-be2c-e658782e36aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:28.659146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be0147-8505-4dc4-bac5-ba490c19b86a None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:28.659678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac49d34-e832-4d32-be2c-e658782e36aa None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:28.659966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac49d34-e832-4d32-be2c-e658782e36aa None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:28.660245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac49d34-e832-4d32-be2c-e658782e36aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:28.660688 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be0147-8505-4dc4-bac5-ba490c19b86a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:28.662459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be0147-8505-4dc4-bac5-ba490c19b86a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:28.664142 dv-cinder-master-3-smb devstack@keystone.service[80376]: WARNING keystone.server.flask.common [None req-d8be0147-8505-4dc4-bac5-ba490c19b86a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 05 10:00:28.670908 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac49d34-e832-4d32-be2c-e658782e36aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:28.672354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac49d34-e832-4d32-be2c-e658782e36aa None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:28.684762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.sql.core [None req-d8be0147-8505-4dc4-bac5-ba490c19b86a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 05 10:00:28.684762 dv-cinder-master-3-smb devstack@keystone.service[80376]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 05 10:00:28.684762 dv-cinder-master-3-smb devstack@keystone.service[80376]: [parameters: {'id': '961e2b2ac7c24e918a8ef21a657433d3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 05 10:00:28.684762 dv-cinder-master-3-smb devstack@keystone.service[80376]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80376) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-3ac49d34-e832-4d32-be2c-e658782e36aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:28.690513 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:28.691466 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:28.692378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:28.692378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:28.692378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:28.692378 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:28.696489 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 460/942] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:28.697773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac49d34-e832-4d32-be2c-e658782e36aa None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-d8be0147-8505-4dc4-bac5-ba490c19b86a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 961e2b2ac7c24e918a8ef21a657433d3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 961e2b2ac7c24e918a8ef21a657433d3. Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application conn.query(q) Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 10:00:28.707474 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application result.read() Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 05 10:00:28.708301 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application next(self.gen) Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application yield resource Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application next(self.gen) Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self.session.rollback() Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self.force_reraise() Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise self.value Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application session.commit() Jul 05 10:00:28.709270 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._prepare_impl() Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self.session.flush() Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._flush(objects) Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application compat.raise_( Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application flush_context.execute() Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 05 10:00:28.710176 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rec.execute(self) Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 05 10:00:28.711109 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application conn.query(q) Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application result.read() Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 10:00:28.713472 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [parameters: {'id': '961e2b2ac7c24e918a8ef21a657433d3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:28.715652 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:28.716665 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:28.717807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:28.717807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 05 10:00:28.717807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 05 10:00:28.717807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:28.717807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:28.717807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 05 10:00:28.717807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 05 10:00:28.717807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 961e2b2ac7c24e918a8ef21a657433d3. Jul 05 10:00:28.717807 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:28.717807 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 483/943] 10.222.0.16 () {60 vars in 1140 bytes} [Tue Jul 5 10:00:28 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:28.717807 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be0147-8505-4dc4-bac5-ba490c19b86a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:28.770567 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:28.843831 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac49d34-e832-4d32-be2c-e658782e36aa None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:28.856545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be0147-8505-4dc4-bac5-ba490c19b86a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:28.880871 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a668a-5b7b-4ec1-9595-5faa9f82e28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:28.881527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a668a-5b7b-4ec1-9595-5faa9f82e28f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:28.881814 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a668a-5b7b-4ec1-9595-5faa9f82e28f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:28.882089 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a668a-5b7b-4ec1-9595-5faa9f82e28f None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:28.882854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a668a-5b7b-4ec1-9595-5faa9f82e28f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:28.883163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a668a-5b7b-4ec1-9595-5faa9f82e28f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:28.885004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a668a-5b7b-4ec1-9595-5faa9f82e28f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:28.888310 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:28.903682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36750cd-10fe-4670-a96b-25d3cc1fa8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:28.904863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36750cd-10fe-4670-a96b-25d3cc1fa8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:28.905483 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36750cd-10fe-4670-a96b-25d3cc1fa8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:28.905740 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 461/944] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:28.906584 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36750cd-10fe-4670-a96b-25d3cc1fa8f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:28.907117 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a668a-5b7b-4ec1-9595-5faa9f82e28f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:28.916862 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36750cd-10fe-4670-a96b-25d3cc1fa8f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:28.919544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36750cd-10fe-4670-a96b-25d3cc1fa8f7 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-e36750cd-10fe-4670-a96b-25d3cc1fa8f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:28.939246 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:28.940233 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:28.941168 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:28.941168 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:28.941168 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:28.941168 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:28.941168 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:28.941168 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:28.941168 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:28.941168 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:28.944141 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 484/945] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:28.950259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36750cd-10fe-4670-a96b-25d3cc1fa8f7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:29.024733 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a668a-5b7b-4ec1-9595-5faa9f82e28f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:29.059545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc15bf-81db-4f44-b74e-38baa05d0e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:29.060354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc15bf-81db-4f44-b74e-38baa05d0e41 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:29.060663 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc15bf-81db-4f44-b74e-38baa05d0e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:29.060977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc15bf-81db-4f44-b74e-38baa05d0e41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:29.068180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc15bf-81db-4f44-b74e-38baa05d0e41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:29.069615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc15bf-81db-4f44-b74e-38baa05d0e41 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:29.074382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:29.078315 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36750cd-10fe-4670-a96b-25d3cc1fa8f7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f5cc15bf-81db-4f44-b74e-38baa05d0e41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:29.087501 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:29.088410 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:29.089311 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:29.089311 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:29.089311 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:29.089311 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:29.089311 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:29.089311 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:29.089311 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:29.089311 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:29.090408 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 462/946] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:29.091722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938660f0-fc9d-4e5d-9ae4-a8e41990dce6 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:29.092011 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938660f0-fc9d-4e5d-9ae4-a8e41990dce6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:29.092291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938660f0-fc9d-4e5d-9ae4-a8e41990dce6 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:29.093586 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 463/947] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:29.095368 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938660f0-fc9d-4e5d-9ae4-a8e41990dce6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:29.122414 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a6c7a-b1c9-4363-be3c-e7cf8334930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:29.123084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a6c7a-b1c9-4363-be3c-e7cf8334930c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:29.123364 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a6c7a-b1c9-4363-be3c-e7cf8334930c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:29.123634 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a6c7a-b1c9-4363-be3c-e7cf8334930c None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:29.124392 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a6c7a-b1c9-4363-be3c-e7cf8334930c None admin] RBAC: Adding query filter params (name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:29.124713 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a6c7a-b1c9-4363-be3c-e7cf8334930c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:29.126154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a6c7a-b1c9-4363-be3c-e7cf8334930c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:29.146099 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 485/948] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:29.147362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87410868-cbd2-4a4a-8c72-f06e67a621ec None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:29.147651 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87410868-cbd2-4a4a-8c72-f06e67a621ec None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:29.147925 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87410868-cbd2-4a4a-8c72-f06e67a621ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:29.165104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:29.165751 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-87410868-cbd2-4a4a-8c72-f06e67a621ec None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:29.167325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-87410868-cbd2-4a4a-8c72-f06e67a621ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:29.230671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938660f0-fc9d-4e5d-9ae4-a8e41990dce6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:29.250688 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 486/949] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:29.267328 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87410868-cbd2-4a4a-8c72-f06e67a621ec None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:29.275176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae6570-7e63-42a5-8718-715bd6ee0dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:29.275869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae6570-7e63-42a5-8718-715bd6ee0dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:29.276844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae6570-7e63-42a5-8718-715bd6ee0dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:29.277135 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae6570-7e63-42a5-8718-715bd6ee0dc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:29.277954 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae6570-7e63-42a5-8718-715bd6ee0dc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:29.278291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae6570-7e63-42a5-8718-715bd6ee0dc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:29.280687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae6570-7e63-42a5-8718-715bd6ee0dc6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:29.298981 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 464/950] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:29.305002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae6570-7e63-42a5-8718-715bd6ee0dc6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:29.394913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:29.411938 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87410868-cbd2-4a4a-8c72-f06e67a621ec None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:29.434788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae6570-7e63-42a5-8718-715bd6ee0dc6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:29.453948 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:29.454933 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219491d4-b598-4011-8317-9c5dfd015df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:29.456223 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219491d4-b598-4011-8317-9c5dfd015df5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:29.457027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219491d4-b598-4011-8317-9c5dfd015df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:29.457544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219491d4-b598-4011-8317-9c5dfd015df5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:29.458378 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219491d4-b598-4011-8317-9c5dfd015df5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:29.460767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219491d4-b598-4011-8317-9c5dfd015df5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:29.474488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032dc6d-ec78-4596-950a-9a14e9121c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-219491d4-b598-4011-8317-9c5dfd015df5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:29.475076 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:29.476277 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:29.477174 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:29.477958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032dc6d-ec78-4596-950a-9a14e9121c85 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:29.477958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032dc6d-ec78-4596-950a-9a14e9121c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:29.477958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032dc6d-ec78-4596-950a-9a14e9121c85 None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/84a8d62afe964cb49acc236850adbbf3/roles/9f8e96f6abc44873a5d208d80eb06f0c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:29.479292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032dc6d-ec78-4596-950a-9a14e9121c85 None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=84a8d62afe964cb49acc236850adbbf3, role_id=9f8e96f6abc44873a5d208d80eb06f0c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:29.480059 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 487/951] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jul 5 10:00:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:29.484122 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032dc6d-ec78-4596-950a-9a14e9121c85 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:29.486572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219491d4-b598-4011-8317-9c5dfd015df5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:29.500552 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 465/952] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 10:00:29 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/84a8d62afe964cb49acc236850adbbf3/roles/9f8e96f6abc44873a5d208d80eb06f0c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:00:29.608398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219491d4-b598-4011-8317-9c5dfd015df5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:29.647394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1beee0-a1b6-4fbb-aee2-e591b9756bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:29.648125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1beee0-a1b6-4fbb-aee2-e591b9756bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:29.648667 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1beee0-a1b6-4fbb-aee2-e591b9756bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:29.649042 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1beee0-a1b6-4fbb-aee2-e591b9756bc1 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:29.649858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1beee0-a1b6-4fbb-aee2-e591b9756bc1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:29.650281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1beee0-a1b6-4fbb-aee2-e591b9756bc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:29.651672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1beee0-a1b6-4fbb-aee2-e591b9756bc1 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:29.657386 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 488/953] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jul 5 10:00:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:29.663327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032dc6d-ec78-4596-950a-9a14e9121c85 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:29.790538 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032dc6d-ec78-4596-950a-9a14e9121c85 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:29.825803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7349718-45ad-44a1-94e0-db7a59c3cc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:29.826868 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7349718-45ad-44a1-94e0-db7a59c3cc8a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:29.827484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7349718-45ad-44a1-94e0-db7a59c3cc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:29.827960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7349718-45ad-44a1-94e0-db7a59c3cc8a None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:29.828895 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7349718-45ad-44a1-94e0-db7a59c3cc8a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:29.829445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7349718-45ad-44a1-94e0-db7a59c3cc8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:29.831156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7349718-45ad-44a1-94e0-db7a59c3cc8a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:29.837363 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 466/954] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jul 5 10:00:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:29.846302 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1beee0-a1b6-4fbb-aee2-e591b9756bc1 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:29.865626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:29.951119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1beee0-a1b6-4fbb-aee2-e591b9756bc1 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:29.986031 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115d956-83e4-427c-8711-386a74f8df0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:29.986743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115d956-83e4-427c-8711-386a74f8df0b None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:29.987374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115d956-83e4-427c-8711-386a74f8df0b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:29.987652 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115d956-83e4-427c-8711-386a74f8df0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:29.988464 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115d956-83e4-427c-8711-386a74f8df0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:29.989903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115d956-83e4-427c-8711-386a74f8df0b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:30.012218 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 489/955] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jul 5 10:00:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:30.183746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95cf46-e45f-44bd-b176-3a4bf333c157 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:30.184048 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95cf46-e45f-44bd-b176-3a4bf333c157 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:30.184370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95cf46-e45f-44bd-b176-3a4bf333c157 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:30.185952 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 467/956] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:30.191424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b858b1-e26a-40b4-9021-fccf4d64a19f None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:30.191973 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b858b1-e26a-40b4-9021-fccf4d64a19f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:30.192468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b858b1-e26a-40b4-9021-fccf4d64a19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:30.210094 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-07b858b1-e26a-40b4-9021-fccf4d64a19f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:30.211585 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-07b858b1-e26a-40b4-9021-fccf4d64a19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:30.277574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:30.288772 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 490/957] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:30.307010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95cf46-e45f-44bd-b176-3a4bf333c157 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:30.353150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:30.416719 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95cf46-e45f-44bd-b176-3a4bf333c157 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:30.451038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed620ef3-7848-409f-be47-4dad1bd92688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:30.451851 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed620ef3-7848-409f-be47-4dad1bd92688 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:30.452135 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed620ef3-7848-409f-be47-4dad1bd92688 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:30.452411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed620ef3-7848-409f-be47-4dad1bd92688 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:30.453071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed620ef3-7848-409f-be47-4dad1bd92688 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:30.454494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed620ef3-7848-409f-be47-4dad1bd92688 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ed620ef3-7848-409f-be47-4dad1bd92688 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:30.466779 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:30.467998 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:30.468954 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 468/958] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:30.474361 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b858b1-e26a-40b4-9021-fccf4d64a19f None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:30.573939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b858b1-e26a-40b4-9021-fccf4d64a19f None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:30.609203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5488ae-eb9e-4561-b3ea-c05509748dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:30.610306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5488ae-eb9e-4561-b3ea-c05509748dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:30.610811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5488ae-eb9e-4561-b3ea-c05509748dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:30.611297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5488ae-eb9e-4561-b3ea-c05509748dd0 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:30.612312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5488ae-eb9e-4561-b3ea-c05509748dd0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:30.612845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5488ae-eb9e-4561-b3ea-c05509748dd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:30.614534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5488ae-eb9e-4561-b3ea-c05509748dd0 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:30.620007 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 491/959] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:30.625918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed620ef3-7848-409f-be47-4dad1bd92688 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:30.726638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed620ef3-7848-409f-be47-4dad1bd92688 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:30.729777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:30.774805 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3dd70-d42e-4c1d-b7ef-580d5b99ad87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:30.775784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3dd70-d42e-4c1d-b7ef-580d5b99ad87 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:30.776275 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3dd70-d42e-4c1d-b7ef-580d5b99ad87 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:30.776753 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3dd70-d42e-4c1d-b7ef-580d5b99ad87 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:30.777488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3dd70-d42e-4c1d-b7ef-580d5b99ad87 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:30.777801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3dd70-d42e-4c1d-b7ef-580d5b99ad87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:30.779215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3dd70-d42e-4c1d-b7ef-580d5b99ad87 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:30.784523 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 469/960] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:31.303814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0866e9-925c-4d3a-b284-fa93e1619feb None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:31.304168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0866e9-925c-4d3a-b284-fa93e1619feb None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:31.304471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0866e9-925c-4d3a-b284-fa93e1619feb None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:31.306144 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 492/961] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:31.311456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea327ef-c7f2-453e-a649-6273482961e8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:31.311747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea327ef-c7f2-453e-a649-6273482961e8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:31.312028 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea327ef-c7f2-453e-a649-6273482961e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:31.328950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3ea327ef-c7f2-453e-a649-6273482961e8 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:31.330151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3ea327ef-c7f2-453e-a649-6273482961e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:31.401266 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 470/962] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:31.425347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0866e9-925c-4d3a-b284-fa93e1619feb None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:31.486944 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:31.526208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0866e9-925c-4d3a-b284-fa93e1619feb None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:31.559905 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc780b-68c2-4bbb-bfd0-b607e5b0cbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:31.560504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc780b-68c2-4bbb-bfd0-b607e5b0cbe1 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:31.560788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc780b-68c2-4bbb-bfd0-b607e5b0cbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:31.561061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc780b-68c2-4bbb-bfd0-b607e5b0cbe1 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:31.561776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc780b-68c2-4bbb-bfd0-b607e5b0cbe1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:31.563243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc780b-68c2-4bbb-bfd0-b607e5b0cbe1 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:31.576408 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.sql.core [None req-87cc780b-68c2-4bbb-bfd0-b607e5b0cbe1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 05 10:00:31.576408 dv-cinder-master-3-smb devstack@keystone.service[80376]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 05 10:00:31.576408 dv-cinder-master-3-smb devstack@keystone.service[80376]: [parameters: {'user_id': '01bd071f93474776badd5e5d5966723a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 05 10:00:31.576408 dv-cinder-master-3-smb devstack@keystone.service[80376]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80376) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-87cc780b-68c2-4bbb-bfd0-b607e5b0cbe1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application conn.query(q) Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application result.read() Jul 05 10:00:31.584446 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 10:00:31.585342 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application next(self.gen) Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application yield resource Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application next(self.gen) Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self.session.rollback() Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self.force_reraise() Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise self.value Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application session.commit() Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 05 10:00:31.586379 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._prepare_impl() Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self.session.flush() Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._flush(objects) Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application compat.raise_( Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application flush_context.execute() Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rec.execute(self) Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 05 10:00:31.587282 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 10:00:31.588196 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application conn.query(q) Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application result.read() Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 10:00:31.589069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [parameters: {'user_id': '01bd071f93474776badd5e5d5966723a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:31.589914 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 05 10:00:31.592365 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 05 10:00:31.594506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 05 10:00:31.594506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 05 10:00:31.594506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 05 10:00:31.594506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 05 10:00:31.594506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 05 10:00:31.594506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:31.594506 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 493/963] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Jul 5 10:00:31 2022] POST /identity/v3/users => generated 160 bytes in 161 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:31.595356 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea327ef-c7f2-453e-a649-6273482961e8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:31.642499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff71357-aad0-4f7a-91e8-3098e14d0476 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:31.642978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff71357-aad0-4f7a-91e8-3098e14d0476 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:31.643159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff71357-aad0-4f7a-91e8-3098e14d0476 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:31.651517 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 494/964] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:31.652891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d258c-f59a-4ad5-9040-3ae44a506fc2 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:31.653181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d258c-f59a-4ad5-9040-3ae44a506fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:31.653737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d258c-f59a-4ad5-9040-3ae44a506fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:31.674429 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-af0d258c-f59a-4ad5-9040-3ae44a506fc2 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:31.675985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-af0d258c-f59a-4ad5-9040-3ae44a506fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:31.708299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:31.718077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea327ef-c7f2-453e-a649-6273482961e8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:31.758395 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 495/965] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:31.763371 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ed72a-9ec0-4981-8e4d-f74d48b730c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:31.764195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ed72a-9ec0-4981-8e4d-f74d48b730c5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:31.764480 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ed72a-9ec0-4981-8e4d-f74d48b730c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:31.766263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ed72a-9ec0-4981-8e4d-f74d48b730c5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:31.773797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ed72a-9ec0-4981-8e4d-f74d48b730c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:31.775282 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ed72a-9ec0-4981-8e4d-f74d48b730c5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:31.786116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d258c-f59a-4ad5-9040-3ae44a506fc2 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ba3ed72a-9ec0-4981-8e4d-f74d48b730c5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:31.796025 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:31.797021 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:31.797996 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:31.797996 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:31.797996 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:31.797996 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:31.797996 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:31.797996 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:31.797996 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:31.797996 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:31.797996 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 10:00:31.797996 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:31.802301 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 471/966] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:00:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:31.803710 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ed72a-9ec0-4981-8e4d-f74d48b730c5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:31.851321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:31.908705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d258c-f59a-4ad5-9040-3ae44a506fc2 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:31.932198 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ed72a-9ec0-4981-8e4d-f74d48b730c5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:31.946255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873abc24-7b84-4743-a587-4a8419e024b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:31.947434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873abc24-7b84-4743-a587-4a8419e024b4 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:31.947918 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873abc24-7b84-4743-a587-4a8419e024b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:31.948400 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873abc24-7b84-4743-a587-4a8419e024b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:31.959895 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873abc24-7b84-4743-a587-4a8419e024b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-873abc24-7b84-4743-a587-4a8419e024b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:31.968573 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:31.969535 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:31.970430 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:31.976721 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 496/967] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jul 5 10:00:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:31.977764 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873abc24-7b84-4743-a587-4a8419e024b4 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:31.981149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8173d-7783-4e6a-8c1c-533be5090318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:31.981839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8173d-7783-4e6a-8c1c-533be5090318 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:31.982120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8173d-7783-4e6a-8c1c-533be5090318 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:31.982433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8173d-7783-4e6a-8c1c-533be5090318 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:31.984238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8173d-7783-4e6a-8c1c-533be5090318 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:31.984554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8173d-7783-4e6a-8c1c-533be5090318 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:31.985999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8173d-7783-4e6a-8c1c-533be5090318 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:32.024973 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 472/968] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jul 5 10:00:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:32.095243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873abc24-7b84-4743-a587-4a8419e024b4 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:32.111000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:32.133355 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7b139-b3f0-4b20-a415-b1c6c975f1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:32.134086 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7b139-b3f0-4b20-a415-b1c6c975f1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.134402 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7b139-b3f0-4b20-a415-b1c6c975f1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.134679 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7b139-b3f0-4b20-a415-b1c6c975f1a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.135436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7b139-b3f0-4b20-a415-b1c6c975f1a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:32.135741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7b139-b3f0-4b20-a415-b1c6c975f1a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:32.137268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7b139-b3f0-4b20-a415-b1c6c975f1a2 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:32.147216 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 497/969] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jul 5 10:00:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:32.155124 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8173d-7783-4e6a-8c1c-533be5090318 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:32.174944 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:32.191352 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e876b-b0fb-4568-a4e6-24142d80e8df None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.191666 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e876b-b0fb-4568-a4e6-24142d80e8df None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.191945 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e876b-b0fb-4568-a4e6-24142d80e8df None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.196134 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 498/970] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:32.198649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f948b-830f-4f21-be83-65ec88972a36 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.198929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f948b-830f-4f21-be83-65ec88972a36 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.199213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f948b-830f-4f21-be83-65ec88972a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.216882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-066f948b-830f-4f21-be83-65ec88972a36 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:32.218123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-066f948b-830f-4f21-be83-65ec88972a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:32.272091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8173d-7783-4e6a-8c1c-533be5090318 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:32.304855 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 499/971] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:32.313312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7d33a-fdc8-4ae9-9919-40b37bbd3197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:32.314119 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7d33a-fdc8-4ae9-9919-40b37bbd3197 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.314732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7d33a-fdc8-4ae9-9919-40b37bbd3197 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.315011 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7d33a-fdc8-4ae9-9919-40b37bbd3197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.326188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7d33a-fdc8-4ae9-9919-40b37bbd3197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:32.327681 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7d33a-fdc8-4ae9-9919-40b37bbd3197 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:32.330784 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f948b-830f-4f21-be83-65ec88972a36 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-a1e7d33a-fdc8-4ae9-9919-40b37bbd3197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:32.350579 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:32.351553 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:32.352439 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:32.352439 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:32.352439 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:32.352439 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:32.352439 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 473/972] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:32.357606 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7d33a-fdc8-4ae9-9919-40b37bbd3197 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:32.425864 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:32.437051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f948b-830f-4f21-be83-65ec88972a36 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:32.444886 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:32.463018 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7d33a-fdc8-4ae9-9919-40b37bbd3197 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:32.473748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12caf1-1d76-4714-9604-31cab3d8623c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:32.474600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12caf1-1d76-4714-9604-31cab3d8623c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.474910 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12caf1-1d76-4714-9604-31cab3d8623c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.475185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12caf1-1d76-4714-9604-31cab3d8623c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.475846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12caf1-1d76-4714-9604-31cab3d8623c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:32.477271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12caf1-1d76-4714-9604-31cab3d8623c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-5e12caf1-1d76-4714-9604-31cab3d8623c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:32.490694 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:32.492458 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:32.493549 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 500/973] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jul 5 10:00:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:32.497721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12caf1-1d76-4714-9604-31cab3d8623c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:32.502079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac3044-f269-47e9-b6a7-706ad106a64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:32.502863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac3044-f269-47e9-b6a7-706ad106a64f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.503151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac3044-f269-47e9-b6a7-706ad106a64f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.503420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac3044-f269-47e9-b6a7-706ad106a64f None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.504185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac3044-f269-47e9-b6a7-706ad106a64f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:32.504497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac3044-f269-47e9-b6a7-706ad106a64f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:32.505861 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac3044-f269-47e9-b6a7-706ad106a64f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:32.520168 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 474/974] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:32.525393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac3044-f269-47e9-b6a7-706ad106a64f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:32.609259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12caf1-1d76-4714-9604-31cab3d8623c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:32.634443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac3044-f269-47e9-b6a7-706ad106a64f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:32.644959 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45d7b6-31d2-4597-95dc-00a21e33a100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:32.645982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45d7b6-31d2-4597-95dc-00a21e33a100 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.646300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45d7b6-31d2-4597-95dc-00a21e33a100 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.646894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45d7b6-31d2-4597-95dc-00a21e33a100 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.647621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45d7b6-31d2-4597-95dc-00a21e33a100 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:32.647980 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45d7b6-31d2-4597-95dc-00a21e33a100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:32.649353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45d7b6-31d2-4597-95dc-00a21e33a100 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:32.658253 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 501/975] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jul 5 10:00:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:32.661546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45d7b6-31d2-4597-95dc-00a21e33a100 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:32.672613 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:32.685056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038fa73-dcc1-4013-aebf-69f6cfa18c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:32.685833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038fa73-dcc1-4013-aebf-69f6cfa18c9d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.686574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038fa73-dcc1-4013-aebf-69f6cfa18c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.686574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038fa73-dcc1-4013-aebf-69f6cfa18c9d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.694997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038fa73-dcc1-4013-aebf-69f6cfa18c9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:32.696486 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038fa73-dcc1-4013-aebf-69f6cfa18c9d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:32.700206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1038fa73-dcc1-4013-aebf-69f6cfa18c9d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:32.712570 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:32.713578 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:32.715345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:32.715345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:32.715345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:32.715345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:32.715345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:32.715345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:32.715345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:32.715345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:32.715345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 10:00:32.715345 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:32.715345 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 475/976] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jul 5 10:00:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:32.719293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038fa73-dcc1-4013-aebf-69f6cfa18c9d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:32.778623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45d7b6-31d2-4597-95dc-00a21e33a100 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:32.817353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddd431-6238-4f66-91c9-7e70b9683869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:32.818051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddd431-6238-4f66-91c9-7e70b9683869 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.818416 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddd431-6238-4f66-91c9-7e70b9683869 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.818711 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddd431-6238-4f66-91c9-7e70b9683869 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.819465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddd431-6238-4f66-91c9-7e70b9683869 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:32.819775 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddd431-6238-4f66-91c9-7e70b9683869 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:32.821543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddd431-6238-4f66-91c9-7e70b9683869 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:32.826541 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 502/977] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jul 5 10:00:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:32.832478 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddd431-6238-4f66-91c9-7e70b9683869 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:32.839548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038fa73-dcc1-4013-aebf-69f6cfa18c9d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:32.873977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84fb0f-a414-4dcf-80d2-e15a3e1aa701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:32.874700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84fb0f-a414-4dcf-80d2-e15a3e1aa701 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.874987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84fb0f-a414-4dcf-80d2-e15a3e1aa701 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.875266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84fb0f-a414-4dcf-80d2-e15a3e1aa701 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.876140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84fb0f-a414-4dcf-80d2-e15a3e1aa701 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:32.876453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84fb0f-a414-4dcf-80d2-e15a3e1aa701 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:32.877917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84fb0f-a414-4dcf-80d2-e15a3e1aa701 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:32.894838 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 476/978] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jul 5 10:00:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:32.896096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d45b86-406e-4e90-967a-ed23fce51e44 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.896383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d45b86-406e-4e90-967a-ed23fce51e44 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.896656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d45b86-406e-4e90-967a-ed23fce51e44 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.898004 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 477/979] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:32.900779 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d45b86-406e-4e90-967a-ed23fce51e44 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:32.929325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:32.933128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddd431-6238-4f66-91c9-7e70b9683869 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:32.946633 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:32.967506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae47ce6-f7af-4751-9eac-8c9e70263328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:32.968163 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae47ce6-f7af-4751-9eac-8c9e70263328 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.968440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae47ce6-f7af-4751-9eac-8c9e70263328 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.968732 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae47ce6-f7af-4751-9eac-8c9e70263328 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.969528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae47ce6-f7af-4751-9eac-8c9e70263328 None admin] RBAC: Adding query filter params (interface=admin, service_id=2c12dc0172a34d17bbaebd8a0eadb271, region_id=RegionOne) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:32.969836 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae47ce6-f7af-4751-9eac-8c9e70263328 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:32.971237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae47ce6-f7af-4751-9eac-8c9e70263328 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:32.976596 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 503/980] 10.222.0.16 () {56 vars in 1249 bytes} [Tue Jul 5 10:00:32 2022] GET /identity/v3/endpoints?service_id=2c12dc0172a34d17bbaebd8a0eadb271&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:32.977573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a5296-2f03-4e0c-a6ac-1efd0d7d6983 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:32.977859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a5296-2f03-4e0c-a6ac-1efd0d7d6983 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:32.978135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a5296-2f03-4e0c-a6ac-1efd0d7d6983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:32.994343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-209a5296-2f03-4e0c-a6ac-1efd0d7d6983 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:32.995528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-209a5296-2f03-4e0c-a6ac-1efd0d7d6983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:33.003818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d45b86-406e-4e90-967a-ed23fce51e44 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:33.041530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b3f8a-a7f8-4f71-a878-de8be62979e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:33.042379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b3f8a-a7f8-4f71-a878-de8be62979e0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:33.042702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b3f8a-a7f8-4f71-a878-de8be62979e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:33.043067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b3f8a-a7f8-4f71-a878-de8be62979e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:33.066159 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 504/981] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:33.070461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a5296-2f03-4e0c-a6ac-1efd0d7d6983 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:33.114905 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b3f8a-a7f8-4f71-a878-de8be62979e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:33.116362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b3f8a-a7f8-4f71-a878-de8be62979e0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:33.156687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:33.159440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:33.165726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a5296-2f03-4e0c-a6ac-1efd0d7d6983 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:33.198921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799bdbd-3bbf-4f71-b930-9caa2638781f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:33.199513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799bdbd-3bbf-4f71-b930-9caa2638781f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:33.199796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799bdbd-3bbf-4f71-b930-9caa2638781f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:33.200072 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799bdbd-3bbf-4f71-b930-9caa2638781f None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:33.200753 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799bdbd-3bbf-4f71-b930-9caa2638781f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:33.202128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799bdbd-3bbf-4f71-b930-9caa2638781f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:33.208312 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 505/982] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jul 5 10:00:33 2022] GET /identity/v3/services => generated 1497 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:00:33.209063 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799bdbd-3bbf-4f71-b930-9caa2638781f None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:33.214288 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 478/983] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:32 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:00:33.220130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b3f8a-a7f8-4f71-a878-de8be62979e0 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:33.327207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799bdbd-3bbf-4f71-b930-9caa2638781f None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:33.340171 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b3f8a-a7f8-4f71-a878-de8be62979e0 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:33.365587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf5daf-f6a1-4d95-adb6-1f8bc992968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:33.366308 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf5daf-f6a1-4d95-adb6-1f8bc992968a None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:33.366603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf5daf-f6a1-4d95-adb6-1f8bc992968a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:33.366901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf5daf-f6a1-4d95-adb6-1f8bc992968a None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:33.367645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf5daf-f6a1-4d95-adb6-1f8bc992968a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:33.369018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf5daf-f6a1-4d95-adb6-1f8bc992968a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:33.378777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8aaf4d-4de6-4afd-b222-ed4de1e988fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:33.379647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8aaf4d-4de6-4afd-b222-ed4de1e988fb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:33.380160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8aaf4d-4de6-4afd-b222-ed4de1e988fb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:33.380288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8aaf4d-4de6-4afd-b222-ed4de1e988fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:33.385650 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 506/984] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 10:00:33 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:33.392402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8aaf4d-4de6-4afd-b222-ed4de1e988fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:33.393942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8aaf4d-4de6-4afd-b222-ed4de1e988fb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:33.403268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1f8aaf4d-4de6-4afd-b222-ed4de1e988fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:33.416367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:33.417517 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:33.418583 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:33.418583 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:33.418583 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:33.418583 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:33.420039 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 479/985] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:33.425758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf5daf-f6a1-4d95-adb6-1f8bc992968a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:33.447428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:33.551484 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf5daf-f6a1-4d95-adb6-1f8bc992968a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:33.587870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf55cf-119d-4c02-94f4-05ec49909d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:33.588574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf55cf-119d-4c02-94f4-05ec49909d75 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:33.588848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf55cf-119d-4c02-94f4-05ec49909d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:33.589125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf55cf-119d-4c02-94f4-05ec49909d75 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:33.589860 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf55cf-119d-4c02-94f4-05ec49909d75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:33.590169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf55cf-119d-4c02-94f4-05ec49909d75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:33.591603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf55cf-119d-4c02-94f4-05ec49909d75 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:33.604847 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 507/986] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:33.611040 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8aaf4d-4de6-4afd-b222-ed4de1e988fb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:33.715073 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8aaf4d-4de6-4afd-b222-ed4de1e988fb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:33.751604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e33288-cd05-4381-850c-a9383d32c502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:33.752638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e33288-cd05-4381-850c-a9383d32c502 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:33.754262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e33288-cd05-4381-850c-a9383d32c502 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:33.754262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e33288-cd05-4381-850c-a9383d32c502 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:33.767818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e33288-cd05-4381-850c-a9383d32c502 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:33.769517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e33288-cd05-4381-850c-a9383d32c502 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-71e33288-cd05-4381-850c-a9383d32c502 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:33.783860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:33.785125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:33.786277 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:33.786277 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:33.786277 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:33.786277 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:33.786277 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:33.786277 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:33.786277 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:33.786277 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:33.787883 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 480/987] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:33.793787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf55cf-119d-4c02-94f4-05ec49909d75 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:33.871322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:33.902701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf55cf-119d-4c02-94f4-05ec49909d75 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:33.938558 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7987354-c008-4ef1-bd6e-083977675f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:33.939402 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7987354-c008-4ef1-bd6e-083977675f60 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:33.939779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7987354-c008-4ef1-bd6e-083977675f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:33.940275 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7987354-c008-4ef1-bd6e-083977675f60 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:33.941146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7987354-c008-4ef1-bd6e-083977675f60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:33.941551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7987354-c008-4ef1-bd6e-083977675f60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:33.943106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7987354-c008-4ef1-bd6e-083977675f60 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:33.955202 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 508/988] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:33.961518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e33288-cd05-4381-850c-a9383d32c502 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:34.020954 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:34.064271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e33288-cd05-4381-850c-a9383d32c502 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:34.098241 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bcbaf-0b12-4b1c-a856-d49a768ccf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:34.099063 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcbaf-0b12-4b1c-a856-d49a768ccf0d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:34.099442 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcbaf-0b12-4b1c-a856-d49a768ccf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:34.099815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcbaf-0b12-4b1c-a856-d49a768ccf0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:34.100748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bcbaf-0b12-4b1c-a856-d49a768ccf0d None admin] RBAC: Adding query filter params (role.id=9f8e96f6abc44873a5d208d80eb06f0c, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=84a8d62afe964cb49acc236850adbbf3) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:34.101167 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bcbaf-0b12-4b1c-a856-d49a768ccf0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:34.102744 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bcbaf-0b12-4b1c-a856-d49a768ccf0d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:34.109149 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 481/989] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:33 2022] GET /identity/v3/role_assignments?user.id=84a8d62afe964cb49acc236850adbbf3&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=9f8e96f6abc44873a5d208d80eb06f0c => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:35.241061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679e1ef-017f-4598-bd3b-e2f4add22602 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:35.241579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679e1ef-017f-4598-bd3b-e2f4add22602 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:35.241579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679e1ef-017f-4598-bd3b-e2f4add22602 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:35.242887 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 509/990] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:35.248398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827a589-2dd0-4087-84b4-f07d47dfe711 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:35.248812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827a589-2dd0-4087-84b4-f07d47dfe711 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:35.249002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827a589-2dd0-4087-84b4-f07d47dfe711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:35.265848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7827a589-2dd0-4087-84b4-f07d47dfe711 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:35.267059 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7827a589-2dd0-4087-84b4-f07d47dfe711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:35.337431 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 482/991] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:35.359666 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679e1ef-017f-4598-bd3b-e2f4add22602 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:35.418184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84df62-8874-4015-9788-1a301cde85ed None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:35.418752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84df62-8874-4015-9788-1a301cde85ed None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:35.419036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84df62-8874-4015-9788-1a301cde85ed None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:35.421126 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 483/992] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:35.426616 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5245f7-eaca-40d5-bc91-b2c0219a9b16 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:35.426913 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5245f7-eaca-40d5-bc91-b2c0219a9b16 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:35.427191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5245f7-eaca-40d5-bc91-b2c0219a9b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:35.449184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5f5245f7-eaca-40d5-bc91-b2c0219a9b16 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:35.450968 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5f5245f7-eaca-40d5-bc91-b2c0219a9b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:35.467601 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679e1ef-017f-4598-bd3b-e2f4add22602 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:35.497817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:35.499426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:35.505001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e1149-ac08-47b0-9ae1-327a81556231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:35.505821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e1149-ac08-47b0-9ae1-327a81556231 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:35.506105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e1149-ac08-47b0-9ae1-327a81556231 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:35.506409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e1149-ac08-47b0-9ae1-327a81556231 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:35.507183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e1149-ac08-47b0-9ae1-327a81556231 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:35.508554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e1149-ac08-47b0-9ae1-327a81556231 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-ba3e1149-ac08-47b0-9ae1-327a81556231 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:35.521163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:35.523293 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 10:00:35.524158 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:35.524955 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 484/993] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:35.525042 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 510/994] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jul 5 10:00:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:35.531667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5245f7-eaca-40d5-bc91-b2c0219a9b16 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:35.546346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e1149-ac08-47b0-9ae1-327a81556231 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:35.637239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5245f7-eaca-40d5-bc91-b2c0219a9b16 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:35.649028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e1149-ac08-47b0-9ae1-327a81556231 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:35.672228 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0528d76-4521-4804-8c0c-e2782fc2b613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:35.672904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528d76-4521-4804-8c0c-e2782fc2b613 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:35.673192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528d76-4521-4804-8c0c-e2782fc2b613 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:35.673472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528d76-4521-4804-8c0c-e2782fc2b613 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:35.674182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0528d76-4521-4804-8c0c-e2782fc2b613 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:35.674590 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0528d76-4521-4804-8c0c-e2782fc2b613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:35.675970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0528d76-4521-4804-8c0c-e2782fc2b613 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:35.680967 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 485/995] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jul 5 10:00:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:35.684732 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bd095-d06d-4e0a-942e-e14481843106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:35.685622 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bd095-d06d-4e0a-942e-e14481843106 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:35.685904 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bd095-d06d-4e0a-942e-e14481843106 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:35.686173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bd095-d06d-4e0a-942e-e14481843106 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:35.686378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0528d76-4521-4804-8c0c-e2782fc2b613 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:35.686854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87bd095-d06d-4e0a-942e-e14481843106 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:35.688215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87bd095-d06d-4e0a-942e-e14481843106 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-b87bd095-d06d-4e0a-942e-e14481843106 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:35.704481 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:35.705608 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:35.707459 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 511/996] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:35.711963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bd095-d06d-4e0a-942e-e14481843106 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:35.738672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:35.782504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:35.793936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0528d76-4521-4804-8c0c-e2782fc2b613 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:35.829065 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd99e9-e5ce-4b84-98a9-0b9b0f09f5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:35.829743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd99e9-e5ce-4b84-98a9-0b9b0f09f5aa None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:35.829848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bd095-d06d-4e0a-942e-e14481843106 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:35.830036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd99e9-e5ce-4b84-98a9-0b9b0f09f5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:35.830343 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd99e9-e5ce-4b84-98a9-0b9b0f09f5aa None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:35.831125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd99e9-e5ce-4b84-98a9-0b9b0f09f5aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:35.831752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd99e9-e5ce-4b84-98a9-0b9b0f09f5aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:35.833125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd99e9-e5ce-4b84-98a9-0b9b0f09f5aa None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:35.838199 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 486/997] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jul 5 10:00:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:35.844010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd99e9-e5ce-4b84-98a9-0b9b0f09f5aa None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:35.865405 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b0346-7ba4-4b86-bcd6-15ff852bb5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:35.866336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b0346-7ba4-4b86-bcd6-15ff852bb5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:35.866768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b0346-7ba4-4b86-bcd6-15ff852bb5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:35.867048 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b0346-7ba4-4b86-bcd6-15ff852bb5f1 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:35.867758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b0346-7ba4-4b86-bcd6-15ff852bb5f1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:35.868108 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b0346-7ba4-4b86-bcd6-15ff852bb5f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:35.869486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b0346-7ba4-4b86-bcd6-15ff852bb5f1 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:35.874398 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 512/998] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:35.879948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b0346-7ba4-4b86-bcd6-15ff852bb5f1 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:35.953863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd99e9-e5ce-4b84-98a9-0b9b0f09f5aa None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:35.965493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:35.982559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b0346-7ba4-4b86-bcd6-15ff852bb5f1 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:35.991705 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce3df6-c8db-4548-945c-1948dd5e695c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:35.992385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce3df6-c8db-4548-945c-1948dd5e695c None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:35.992669 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce3df6-c8db-4548-945c-1948dd5e695c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:35.993032 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce3df6-c8db-4548-945c-1948dd5e695c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:35.993797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce3df6-c8db-4548-945c-1948dd5e695c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:35.995197 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce3df6-c8db-4548-945c-1948dd5e695c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:36.006163 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:36.014440 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 487/999] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jul 5 10:00:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:00:36.033079 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19559413-201b-41c4-8f75-2d674c0124b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:36.033835 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19559413-201b-41c4-8f75-2d674c0124b8 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:36.034116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19559413-201b-41c4-8f75-2d674c0124b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:36.034424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19559413-201b-41c4-8f75-2d674c0124b8 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:36.035148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19559413-201b-41c4-8f75-2d674c0124b8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:36.035450 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19559413-201b-41c4-8f75-2d674c0124b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:36.036876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19559413-201b-41c4-8f75-2d674c0124b8 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:36.047493 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 513/1000] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:36.048902 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce3df6-c8db-4548-945c-1948dd5e695c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:36.152377 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a334b43-36d7-4cf9-bfd0-8dea39709324 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:36.152741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a334b43-36d7-4cf9-bfd0-8dea39709324 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:36.152916 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a334b43-36d7-4cf9-bfd0-8dea39709324 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:36.154646 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 514/1001] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:36.159602 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce3df6-c8db-4548-945c-1948dd5e695c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:36.160183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1273fb-66ea-47d4-b8b2-cd948c05792a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:36.160559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1273fb-66ea-47d4-b8b2-cd948c05792a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:36.160905 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1273fb-66ea-47d4-b8b2-cd948c05792a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:36.179783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ca1273fb-66ea-47d4-b8b2-cd948c05792a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:36.181081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ca1273fb-66ea-47d4-b8b2-cd948c05792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:36.196213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c4466-aee6-4370-ba3d-2895f3907d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:36.196916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c4466-aee6-4370-ba3d-2895f3907d95 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:36.197197 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c4466-aee6-4370-ba3d-2895f3907d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:36.197458 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c4466-aee6-4370-ba3d-2895f3907d95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:36.198298 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c4466-aee6-4370-ba3d-2895f3907d95 None admin] RBAC: Adding query filter params (interface=public, service_id=cb52d365a8b441098bef21feff7e350d, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:36.198614 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c4466-aee6-4370-ba3d-2895f3907d95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:36.200026 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c4466-aee6-4370-ba3d-2895f3907d95 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:36.207397 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 488/1002] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jul 5 10:00:36 2022] GET /identity/v3/endpoints?service_id=cb52d365a8b441098bef21feff7e350d&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:36.213249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c4466-aee6-4370-ba3d-2895f3907d95 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:36.249489 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:36.263126 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 515/1003] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:36.285368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1273fb-66ea-47d4-b8b2-cd948c05792a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:36.319925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c4466-aee6-4370-ba3d-2895f3907d95 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:36.354285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07d176-3f7d-42ba-9da7-23019abc48d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:36.354956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07d176-3f7d-42ba-9da7-23019abc48d5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:36.355237 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07d176-3f7d-42ba-9da7-23019abc48d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:36.355549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07d176-3f7d-42ba-9da7-23019abc48d5 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:36.356231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07d176-3f7d-42ba-9da7-23019abc48d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:36.357592 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07d176-3f7d-42ba-9da7-23019abc48d5 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:36.374251 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 489/1004] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jul 5 10:00:36 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:00:36.395805 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1273fb-66ea-47d4-b8b2-cd948c05792a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:36.411652 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:36.436627 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3a1fb-9b0b-4f52-9c60-73da3114d41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:36.437450 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3a1fb-9b0b-4f52-9c60-73da3114d41d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:36.437737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3a1fb-9b0b-4f52-9c60-73da3114d41d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:36.438017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3a1fb-9b0b-4f52-9c60-73da3114d41d None admin] PATH_INFO: `/v3/services/image` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:36.438717 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3a1fb-9b0b-4f52-9c60-73da3114d41d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:36.440169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3a1fb-9b0b-4f52-9c60-73da3114d41d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-c3b3a1fb-9b0b-4f52-9c60-73da3114d41d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:36.452715 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:36.453785 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 10:00:36.455244 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:36.458629 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 516/1005] 10.222.0.16 () {56 vars in 1105 bytes} [Tue Jul 5 10:00:36 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:36.460401 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07d176-3f7d-42ba-9da7-23019abc48d5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:36.565038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:36.570963 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07d176-3f7d-42ba-9da7-23019abc48d5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:36.606028 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f8031-4942-4339-b194-771272ae421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:36.606751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f8031-4942-4339-b194-771272ae421b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:36.607039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f8031-4942-4339-b194-771272ae421b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:36.607314 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f8031-4942-4339-b194-771272ae421b None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:36.608040 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f8031-4942-4339-b194-771272ae421b None admin] RBAC: Adding query filter params (name=image) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:36.608349 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f8031-4942-4339-b194-771272ae421b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:36.609780 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f8031-4942-4339-b194-771272ae421b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:36.619512 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 490/1006] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:36.621057 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3a1fb-9b0b-4f52-9c60-73da3114d41d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:36.725157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3a1fb-9b0b-4f52-9c60-73da3114d41d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:36.768498 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090d1a1-3238-4021-bba7-3786a3f09a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:36.769272 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090d1a1-3238-4021-bba7-3786a3f09a85 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:36.769555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090d1a1-3238-4021-bba7-3786a3f09a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:36.769881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090d1a1-3238-4021-bba7-3786a3f09a85 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:36.770659 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090d1a1-3238-4021-bba7-3786a3f09a85 None admin] RBAC: Adding query filter params (type=image) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:36.770961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090d1a1-3238-4021-bba7-3786a3f09a85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:36.772355 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090d1a1-3238-4021-bba7-3786a3f09a85 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:36.786256 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 517/1007] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:38.076388 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e028504-96c2-445c-a793-5ac8e8bf37a4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:38.076920 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e028504-96c2-445c-a793-5ac8e8bf37a4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:38.076920 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e028504-96c2-445c-a793-5ac8e8bf37a4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:38.078523 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 491/1008] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:38.084337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4d36a-92f1-430c-ace6-53fd6859a758 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:38.084758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4d36a-92f1-430c-ace6-53fd6859a758 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:38.084866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4d36a-92f1-430c-ace6-53fd6859a758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:38.101844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-2cb4d36a-92f1-430c-ace6-53fd6859a758 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:38.103061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2cb4d36a-92f1-430c-ace6-53fd6859a758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:38.140476 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:38.173096 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 518/1009] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:38.196254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e028504-96c2-445c-a793-5ac8e8bf37a4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:38.299129 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e028504-96c2-445c-a793-5ac8e8bf37a4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:38.333286 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e825a-acff-4028-a244-bfd03210447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:38.334005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e825a-acff-4028-a244-bfd03210447f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:38.334310 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e825a-acff-4028-a244-bfd03210447f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:38.334585 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e825a-acff-4028-a244-bfd03210447f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:38.335768 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e825a-acff-4028-a244-bfd03210447f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:38.337207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e825a-acff-4028-a244-bfd03210447f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:38.338997 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 492/1010] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:38.344606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4d36a-92f1-430c-ace6-53fd6859a758 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:38.402796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f770a8d-ae45-4493-8c68-58cace8b3b6d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:38.403039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f770a8d-ae45-4493-8c68-58cace8b3b6d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:38.403332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f770a8d-ae45-4493-8c68-58cace8b3b6d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:38.404713 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 493/1011] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:38.409561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98478cb0-85a3-40fa-902a-1a7c61140e19 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:38.409838 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98478cb0-85a3-40fa-902a-1a7c61140e19 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:38.410127 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98478cb0-85a3-40fa-902a-1a7c61140e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:38.426767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-98478cb0-85a3-40fa-902a-1a7c61140e19 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:38.428044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-98478cb0-85a3-40fa-902a-1a7c61140e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:38.452718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4d36a-92f1-430c-ace6-53fd6859a758 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:38.453881 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:38.490989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240bfe4-72e5-4cd2-9287-a63b9754d634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:38.491653 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240bfe4-72e5-4cd2-9287-a63b9754d634 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:38.491936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240bfe4-72e5-4cd2-9287-a63b9754d634 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:38.492209 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240bfe4-72e5-4cd2-9287-a63b9754d634 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:38.492913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240bfe4-72e5-4cd2-9287-a63b9754d634 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:38.494379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240bfe4-72e5-4cd2-9287-a63b9754d634 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:38.502279 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 494/1012] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:38.525503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98478cb0-85a3-40fa-902a-1a7c61140e19 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:38.526622 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 519/1013] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 10:00:38 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:38.637887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98478cb0-85a3-40fa-902a-1a7c61140e19 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:38.677145 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd791c6-5761-4aa1-ab9d-52a89d15ea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:38.677964 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd791c6-5761-4aa1-ab9d-52a89d15ea6c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:38.678276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd791c6-5761-4aa1-ab9d-52a89d15ea6c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:38.678561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd791c6-5761-4aa1-ab9d-52a89d15ea6c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:38.679231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd791c6-5761-4aa1-ab9d-52a89d15ea6c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:38.680602 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd791c6-5761-4aa1-ab9d-52a89d15ea6c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-dcd791c6-5761-4aa1-ab9d-52a89d15ea6c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:38.692727 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:38.693788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 10:00:38.694795 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:38.696997 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 495/1014] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:38.703333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240bfe4-72e5-4cd2-9287-a63b9754d634 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:38.732402 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:38.772101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7dc65-0837-47e3-8307-100337ffc35f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:38.772520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7dc65-0837-47e3-8307-100337ffc35f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:38.772807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7dc65-0837-47e3-8307-100337ffc35f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:38.774324 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 496/1015] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:38.779086 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca878f6-4605-4069-85d6-5324a4f3fd6d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:38.779378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca878f6-4605-4069-85d6-5324a4f3fd6d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:38.779653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca878f6-4605-4069-85d6-5324a4f3fd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:38.797810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5ca878f6-4605-4069-85d6-5324a4f3fd6d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:38.799239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5ca878f6-4605-4069-85d6-5324a4f3fd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:38.816727 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240bfe4-72e5-4cd2-9287-a63b9754d634 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:38.817618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:38.852755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04af00a-8dad-4764-9f89-10b3cc9676f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:38.853446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04af00a-8dad-4764-9f89-10b3cc9676f8 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:38.853737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04af00a-8dad-4764-9f89-10b3cc9676f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:38.854015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04af00a-8dad-4764-9f89-10b3cc9676f8 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:38.854794 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04af00a-8dad-4764-9f89-10b3cc9676f8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:38.855105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04af00a-8dad-4764-9f89-10b3cc9676f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:38.856495 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04af00a-8dad-4764-9f89-10b3cc9676f8 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:38.862459 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 520/1016] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:38.867694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04af00a-8dad-4764-9f89-10b3cc9676f8 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:38.876586 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 497/1017] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:38.899624 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca878f6-4605-4069-85d6-5324a4f3fd6d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:38.963903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:38.979842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04af00a-8dad-4764-9f89-10b3cc9676f8 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:39.007524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca878f6-4605-4069-85d6-5324a4f3fd6d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:39.015861 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252b14f-99f9-4650-bd61-d64843a99813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:39.016537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252b14f-99f9-4650-bd61-d64843a99813 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:39.016818 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252b14f-99f9-4650-bd61-d64843a99813 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:39.017088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252b14f-99f9-4650-bd61-d64843a99813 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:39.017823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252b14f-99f9-4650-bd61-d64843a99813 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:39.018213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252b14f-99f9-4650-bd61-d64843a99813 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:39.019629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252b14f-99f9-4650-bd61-d64843a99813 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:39.024659 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 521/1018] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jul 5 10:00:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:39.030418 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252b14f-99f9-4650-bd61-d64843a99813 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:39.049445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b05bb-671d-4b21-880e-545770458be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:39.050452 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b05bb-671d-4b21-880e-545770458be1 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:39.050892 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b05bb-671d-4b21-880e-545770458be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:39.051304 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b05bb-671d-4b21-880e-545770458be1 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:39.052229 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b05bb-671d-4b21-880e-545770458be1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:39.053740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b05bb-671d-4b21-880e-545770458be1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:39.073161 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 498/1019] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 10:00:38 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:00:39.143352 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252b14f-99f9-4650-bd61-d64843a99813 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:39.185669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262911a0-8186-4411-bd41-15ac48cbd78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:39.186614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262911a0-8186-4411-bd41-15ac48cbd78d None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:39.187025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262911a0-8186-4411-bd41-15ac48cbd78d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:39.187422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262911a0-8186-4411-bd41-15ac48cbd78d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:39.188551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262911a0-8186-4411-bd41-15ac48cbd78d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:39.190913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262911a0-8186-4411-bd41-15ac48cbd78d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:39.215087 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 522/1020] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jul 5 10:00:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:00:40.590978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72949dc-e3e4-474a-83ad-f4d077a38d57 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:40.591542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72949dc-e3e4-474a-83ad-f4d077a38d57 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:40.591542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72949dc-e3e4-474a-83ad-f4d077a38d57 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:40.593110 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 499/1021] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:40.598424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71234e0a-b206-4def-bcfc-0ce80db984e4 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:40.598837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71234e0a-b206-4def-bcfc-0ce80db984e4 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:40.599003 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71234e0a-b206-4def-bcfc-0ce80db984e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:40.606392 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:40.617333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-71234e0a-b206-4def-bcfc-0ce80db984e4 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:40.618963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-71234e0a-b206-4def-bcfc-0ce80db984e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:40.696447 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 523/1022] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:40.720923 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72949dc-e3e4-474a-83ad-f4d077a38d57 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:40.737907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:40.834535 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72949dc-e3e4-474a-83ad-f4d077a38d57 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:40.870969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f3136-4f8e-45d9-a198-d01ca7ab3291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:40.871701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f3136-4f8e-45d9-a198-d01ca7ab3291 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:40.871985 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f3136-4f8e-45d9-a198-d01ca7ab3291 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:40.872258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f3136-4f8e-45d9-a198-d01ca7ab3291 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:40.873448 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f3136-4f8e-45d9-a198-d01ca7ab3291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:40.874940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f3136-4f8e-45d9-a198-d01ca7ab3291 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:40.876789 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 500/1023] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:40.882448 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71234e0a-b206-4def-bcfc-0ce80db984e4 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:40.982854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71234e0a-b206-4def-bcfc-0ce80db984e4 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:41.011658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:41.018989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778d160-0fec-4d75-9274-e8b86b792337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:41.019670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778d160-0fec-4d75-9274-e8b86b792337 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.019950 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778d160-0fec-4d75-9274-e8b86b792337 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.020226 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778d160-0fec-4d75-9274-e8b86b792337 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.020944 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778d160-0fec-4d75-9274-e8b86b792337 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:41.022392 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778d160-0fec-4d75-9274-e8b86b792337 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:41.046256 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 524/1024] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 10:00:40 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:41.136329 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c7568-6394-4273-b231-d457bd263d72 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.136575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c7568-6394-4273-b231-d457bd263d72 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.136900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c7568-6394-4273-b231-d457bd263d72 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.138162 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 501/1025] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:41.144355 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac9c56-35bc-4986-a1b0-f96b8361369c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.144659 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac9c56-35bc-4986-a1b0-f96b8361369c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.144951 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac9c56-35bc-4986-a1b0-f96b8361369c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.163774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-faac9c56-35bc-4986-a1b0-f96b8361369c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:41.164994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-faac9c56-35bc-4986-a1b0-f96b8361369c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:41.243042 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 525/1026] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:41.265286 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c7568-6394-4273-b231-d457bd263d72 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:41.296243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5276b-be5e-445a-8d1a-7e304ff0851b None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.296529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5276b-be5e-445a-8d1a-7e304ff0851b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.296804 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5276b-be5e-445a-8d1a-7e304ff0851b None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.298193 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 526/1027] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:41.304225 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c43f6-07d9-4d1b-a164-02e88f886901 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.304511 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c43f6-07d9-4d1b-a164-02e88f886901 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.304785 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c43f6-07d9-4d1b-a164-02e88f886901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.321468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-c19c43f6-07d9-4d1b-a164-02e88f886901 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:41.322677 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c19c43f6-07d9-4d1b-a164-02e88f886901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:41.354448 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:41.375105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c7568-6394-4273-b231-d457bd263d72 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:41.392465 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 527/1028] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:41.410912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc03b5d-96f0-462f-aa9f-d76f78e76f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:41.411714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc03b5d-96f0-462f-aa9f-d76f78e76f7a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.411996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc03b5d-96f0-462f-aa9f-d76f78e76f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.412269 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc03b5d-96f0-462f-aa9f-d76f78e76f7a None admin] PATH_INFO: `/v3/services/image` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.412917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc03b5d-96f0-462f-aa9f-d76f78e76f7a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:41.414306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc03b5d-96f0-462f-aa9f-d76f78e76f7a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:41.416717 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c43f6-07d9-4d1b-a164-02e88f886901 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-dcc03b5d-96f0-462f-aa9f-d76f78e76f7a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:41.426162 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:41.427477 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:41.429062 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 502/1029] 10.222.0.16 () {56 vars in 1105 bytes} [Tue Jul 5 10:00:41 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:41.433463 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc03b5d-96f0-462f-aa9f-d76f78e76f7a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:41.471444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:41.531036 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c43f6-07d9-4d1b-a164-02e88f886901 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:41.544180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc03b5d-96f0-462f-aa9f-d76f78e76f7a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:41.568875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3fa090-dada-4791-9f87-50bc0ca0e330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:41.569707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fa090-dada-4791-9f87-50bc0ca0e330 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.569994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fa090-dada-4791-9f87-50bc0ca0e330 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.570281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fa090-dada-4791-9f87-50bc0ca0e330 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.580976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1d51d-bf14-44c2-a6ea-35ecbe4eaf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:41.580976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1d51d-bf14-44c2-a6ea-35ecbe4eaf3b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.580976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1d51d-bf14-44c2-a6ea-35ecbe4eaf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.580976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1d51d-bf14-44c2-a6ea-35ecbe4eaf3b None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.581721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3fa090-dada-4791-9f87-50bc0ca0e330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:41.581960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1d51d-bf14-44c2-a6ea-35ecbe4eaf3b None admin] RBAC: Adding query filter params (name=image) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:41.582273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1d51d-bf14-44c2-a6ea-35ecbe4eaf3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:41.583698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1d51d-bf14-44c2-a6ea-35ecbe4eaf3b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:41.593710 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 503/1030] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-bf3fa090-dada-4791-9f87-50bc0ca0e330 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:41.594313 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:41.595431 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:41.596831 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 528/1031] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jul 5 10:00:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:41.598478 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1d51d-bf14-44c2-a6ea-35ecbe4eaf3b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:41.598622 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3fa090-dada-4791-9f87-50bc0ca0e330 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:41.611513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:41.703121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3fa090-dada-4791-9f87-50bc0ca0e330 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:41.703458 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1d51d-bf14-44c2-a6ea-35ecbe4eaf3b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:41.710710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:41.739646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535d318-2687-446c-8714-07a2f9d1f975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:41.740098 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c27c6-e32d-43ee-9249-c90a28df4add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:41.740328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535d318-2687-446c-8714-07a2f9d1f975 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.740598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535d318-2687-446c-8714-07a2f9d1f975 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.740760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c27c6-e32d-43ee-9249-c90a28df4add None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.740868 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535d318-2687-446c-8714-07a2f9d1f975 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.741041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c27c6-e32d-43ee-9249-c90a28df4add None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.741312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c27c6-e32d-43ee-9249-c90a28df4add None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.741575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535d318-2687-446c-8714-07a2f9d1f975 None admin] RBAC: Adding query filter params (type=image) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:41.741885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535d318-2687-446c-8714-07a2f9d1f975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:41.742060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c27c6-e32d-43ee-9249-c90a28df4add None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:41.742399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c27c6-e32d-43ee-9249-c90a28df4add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:41.743288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535d318-2687-446c-8714-07a2f9d1f975 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:41.743762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c27c6-e32d-43ee-9249-c90a28df4add None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:41.753285 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 504/1032] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:41.759445 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 529/1033] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jul 5 10:00:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:41.759588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535d318-2687-446c-8714-07a2f9d1f975 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:41.762850 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c27c6-e32d-43ee-9249-c90a28df4add None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:41.845430 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:41.867203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c27c6-e32d-43ee-9249-c90a28df4add None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:41.872854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535d318-2687-446c-8714-07a2f9d1f975 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:41.902210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c10fc2-11e0-46bc-86ce-9f45c7d07337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:41.903015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c10fc2-11e0-46bc-86ce-9f45c7d07337 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.903296 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c10fc2-11e0-46bc-86ce-9f45c7d07337 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.903580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c10fc2-11e0-46bc-86ce-9f45c7d07337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.907400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6eee0-22c6-44e3-8225-08ac0cb6c4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:41.908058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6eee0-22c6-44e3-8225-08ac0cb6c4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:41.908390 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6eee0-22c6-44e3-8225-08ac0cb6c4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:41.908668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6eee0-22c6-44e3-8225-08ac0cb6c4d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:41.909457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6eee0-22c6-44e3-8225-08ac0cb6c4d8 None admin] RBAC: Adding query filter params (interface=public, service_id=b6cb956c8b3d46c18914f5cd656b5a1a, region_id=RegionOne) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:41.909770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6eee0-22c6-44e3-8225-08ac0cb6c4d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:41.911168 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6eee0-22c6-44e3-8225-08ac0cb6c4d8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:41.913977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c10fc2-11e0-46bc-86ce-9f45c7d07337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:41.915417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c10fc2-11e0-46bc-86ce-9f45c7d07337 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:41.916492 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 505/1034] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jul 5 10:00:41 2022] GET /identity/v3/endpoints?service_id=b6cb956c8b3d46c18914f5cd656b5a1a&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:41.921632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6eee0-22c6-44e3-8225-08ac0cb6c4d8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-56c10fc2-11e0-46bc-86ce-9f45c7d07337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:41.932522 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:41.933423 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:41.934453 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:41.934453 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:41.934453 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:41.934453 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:41.934453 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 530/1035] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:41.939231 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c10fc2-11e0-46bc-86ce-9f45c7d07337 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:41.963254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:42.036037 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6eee0-22c6-44e3-8225-08ac0cb6c4d8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:42.041582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c10fc2-11e0-46bc-86ce-9f45c7d07337 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:42.073970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406edbb7-ed35-4c1e-b5b2-d9b8f5625a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:42.074685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406edbb7-ed35-4c1e-b5b2-d9b8f5625a40 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:42.074962 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406edbb7-ed35-4c1e-b5b2-d9b8f5625a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:42.075246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406edbb7-ed35-4c1e-b5b2-d9b8f5625a40 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:42.075930 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406edbb7-ed35-4c1e-b5b2-d9b8f5625a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:42.076031 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9924d1-5926-4709-a2d1-8a02eed64f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:42.076653 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9924d1-5926-4709-a2d1-8a02eed64f35 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:42.076934 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9924d1-5926-4709-a2d1-8a02eed64f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:42.077205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9924d1-5926-4709-a2d1-8a02eed64f35 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:42.077299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406edbb7-ed35-4c1e-b5b2-d9b8f5625a40 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:42.077919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9924d1-5926-4709-a2d1-8a02eed64f35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:42.078389 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9924d1-5926-4709-a2d1-8a02eed64f35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:42.079606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9924d1-5926-4709-a2d1-8a02eed64f35 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:42.092181 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 506/1036] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jul 5 10:00:41 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:00:42.092382 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 531/1037] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:42.097579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406edbb7-ed35-4c1e-b5b2-d9b8f5625a40 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:42.100571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:42.217412 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406edbb7-ed35-4c1e-b5b2-d9b8f5625a40 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:42.256141 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74948f26-b0b6-4d19-ab8e-c69a8fe25956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:42.256943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74948f26-b0b6-4d19-ab8e-c69a8fe25956 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:42.257231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74948f26-b0b6-4d19-ab8e-c69a8fe25956 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:42.257500 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74948f26-b0b6-4d19-ab8e-c69a8fe25956 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:42.265508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74948f26-b0b6-4d19-ab8e-c69a8fe25956 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:42.267198 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74948f26-b0b6-4d19-ab8e-c69a8fe25956 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-74948f26-b0b6-4d19-ab8e-c69a8fe25956 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:42.281407 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:42.282434 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:42.283526 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:42.283526 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:42.283526 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:42.283526 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:42.283526 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:42.283526 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:42.283526 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:42.283526 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:42.283526 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:42.283526 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:42.283526 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 507/1038] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jul 5 10:00:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:42.288500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9924d1-5926-4709-a2d1-8a02eed64f35 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:42.370889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:42.391650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9924d1-5926-4709-a2d1-8a02eed64f35 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:42.425555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc0e8b-ff57-40bb-9a5d-68695313c752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:42.426362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc0e8b-ff57-40bb-9a5d-68695313c752 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:42.426651 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc0e8b-ff57-40bb-9a5d-68695313c752 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:42.426934 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc0e8b-ff57-40bb-9a5d-68695313c752 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:42.427774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc0e8b-ff57-40bb-9a5d-68695313c752 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:42.428084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc0e8b-ff57-40bb-9a5d-68695313c752 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:42.429530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc0e8b-ff57-40bb-9a5d-68695313c752 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:42.447208 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 532/1039] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jul 5 10:00:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:42.452902 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74948f26-b0b6-4d19-ab8e-c69a8fe25956 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:42.514154 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:42.554994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74948f26-b0b6-4d19-ab8e-c69a8fe25956 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:42.588770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaacec-1bf9-48a9-8964-ae5a2cd742cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:42.589432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaacec-1bf9-48a9-8964-ae5a2cd742cc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:42.589721 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaacec-1bf9-48a9-8964-ae5a2cd742cc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:42.590001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaacec-1bf9-48a9-8964-ae5a2cd742cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:42.656471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aaacec-1bf9-48a9-8964-ae5a2cd742cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:42.657926 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aaacec-1bf9-48a9-8964-ae5a2cd742cc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:42.719369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:42.769521 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 508/1040] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:42 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:00:42.775247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc0e8b-ff57-40bb-9a5d-68695313c752 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:42.879874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc0e8b-ff57-40bb-9a5d-68695313c752 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:42.915294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d837c9-0920-402f-ae47-d5e411a58e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:42.916098 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d837c9-0920-402f-ae47-d5e411a58e89 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:42.916381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d837c9-0920-402f-ae47-d5e411a58e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:42.916661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d837c9-0920-402f-ae47-d5e411a58e89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:42.923289 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:42.928143 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d837c9-0920-402f-ae47-d5e411a58e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:42.929613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d837c9-0920-402f-ae47-d5e411a58e89 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-87d837c9-0920-402f-ae47-d5e411a58e89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:42.947011 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:42.947877 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:42.948772 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:42.948772 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:42.948772 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:42.948772 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:42.950310 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 533/1041] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:42.953893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaacec-1bf9-48a9-8964-ae5a2cd742cc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:43.055583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaacec-1bf9-48a9-8964-ae5a2cd742cc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:43.090023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39dad5-3f6f-4bfa-8d0a-644a757100d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:43.090726 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39dad5-3f6f-4bfa-8d0a-644a757100d6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:43.091013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39dad5-3f6f-4bfa-8d0a-644a757100d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:43.091287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39dad5-3f6f-4bfa-8d0a-644a757100d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:43.092007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39dad5-3f6f-4bfa-8d0a-644a757100d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:43.092319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39dad5-3f6f-4bfa-8d0a-644a757100d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:43.093687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39dad5-3f6f-4bfa-8d0a-644a757100d6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:43.105367 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 509/1042] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:43.110808 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d837c9-0920-402f-ae47-d5e411a58e89 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:43.127050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575023-e545-4e12-a672-c2baf853f80b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:43.127528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575023-e545-4e12-a672-c2baf853f80b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:43.127821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575023-e545-4e12-a672-c2baf853f80b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:43.129187 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 510/1043] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:43.134345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f983d-a6d7-4f31-ac73-8f3d8c7f8eba None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:43.134638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f983d-a6d7-4f31-ac73-8f3d8c7f8eba None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:43.134918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f983d-a6d7-4f31-ac73-8f3d8c7f8eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:43.151528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b07f983d-a6d7-4f31-ac73-8f3d8c7f8eba None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:43.152717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b07f983d-a6d7-4f31-ac73-8f3d8c7f8eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:43.169515 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:43.221873 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d837c9-0920-402f-ae47-d5e411a58e89 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:43.231898 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 511/1044] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:43.255179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f983d-a6d7-4f31-ac73-8f3d8c7f8eba None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:43.259359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faf3f4-1f6f-4d30-8801-90728b27fc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:43.260201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faf3f4-1f6f-4d30-8801-90728b27fc88 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:43.260487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faf3f4-1f6f-4d30-8801-90728b27fc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:43.260762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12faf3f4-1f6f-4d30-8801-90728b27fc88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:43.268956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12faf3f4-1f6f-4d30-8801-90728b27fc88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:43.270446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12faf3f4-1f6f-4d30-8801-90728b27fc88 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-12faf3f4-1f6f-4d30-8801-90728b27fc88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:43.284165 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:43.285054 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:43.285971 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:43.285971 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:43.285971 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:43.285971 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:43.285971 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:43.285971 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:43.285971 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:43.285971 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:43.285971 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 534/1045] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:43.291000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faf3f4-1f6f-4d30-8801-90728b27fc88 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:43.344157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:43.362770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f983d-a6d7-4f31-ac73-8f3d8c7f8eba None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:43.396607 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f4674-caf6-4068-8e9d-35982296fbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:43.397400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f4674-caf6-4068-8e9d-35982296fbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:43.397613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12faf3f4-1f6f-4d30-8801-90728b27fc88 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:43.397698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f4674-caf6-4068-8e9d-35982296fbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:43.397946 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f4674-caf6-4068-8e9d-35982296fbe6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:43.408439 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f4674-caf6-4068-8e9d-35982296fbe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-804f4674-caf6-4068-8e9d-35982296fbe6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:43.416146 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:43.417065 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:43.419093 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 512/1046] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:43.423202 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f4674-caf6-4068-8e9d-35982296fbe6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:43.436971 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9153c4f-44df-46c8-ae86-1baea6637bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:43.437640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9153c4f-44df-46c8-ae86-1baea6637bdf None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:43.437927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9153c4f-44df-46c8-ae86-1baea6637bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:43.438207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9153c4f-44df-46c8-ae86-1baea6637bdf None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:43.439045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9153c4f-44df-46c8-ae86-1baea6637bdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:43.439232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:43.439397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9153c4f-44df-46c8-ae86-1baea6637bdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:43.440946 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9153c4f-44df-46c8-ae86-1baea6637bdf None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:43.458516 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 535/1047] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:43.460013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9153c4f-44df-46c8-ae86-1baea6637bdf None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:43.530652 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f4674-caf6-4068-8e9d-35982296fbe6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:43.557600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9153c4f-44df-46c8-ae86-1baea6637bdf None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:43.568261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:43.570578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01415a51-b86c-4461-8366-216432047726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:43.571508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01415a51-b86c-4461-8366-216432047726 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:43.571947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01415a51-b86c-4461-8366-216432047726 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:43.572381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01415a51-b86c-4461-8366-216432047726 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:43.573299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01415a51-b86c-4461-8366-216432047726 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:43.573775 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01415a51-b86c-4461-8366-216432047726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:43.575360 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01415a51-b86c-4461-8366-216432047726 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:43.584704 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 513/1048] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:00:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:43.590055 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01415a51-b86c-4461-8366-216432047726 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:43.592631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef4de-19dd-4bb1-a481-2d2122bb5d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:43.593298 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef4de-19dd-4bb1-a481-2d2122bb5d55 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:43.593579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef4de-19dd-4bb1-a481-2d2122bb5d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:43.593851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef4de-19dd-4bb1-a481-2d2122bb5d55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:43.594768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef4de-19dd-4bb1-a481-2d2122bb5d55 None admin] RBAC: Adding query filter params (role.id=9f8e96f6abc44873a5d208d80eb06f0c, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=fee6ee8f99c34aa281bf02ecd3f23216) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:43.595026 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef4de-19dd-4bb1-a481-2d2122bb5d55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:43.596533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef4de-19dd-4bb1-a481-2d2122bb5d55 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:43.603198 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 536/1049] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:43 2022] GET /identity/v3/role_assignments?user.id=fee6ee8f99c34aa281bf02ecd3f23216&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=9f8e96f6abc44873a5d208d80eb06f0c => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:43.675468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:43.703690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01415a51-b86c-4461-8366-216432047726 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:43.740530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43579a8f-a4de-4fc2-8dab-2939b86b7d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:43.741716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43579a8f-a4de-4fc2-8dab-2939b86b7d0d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:43.742149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43579a8f-a4de-4fc2-8dab-2939b86b7d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:43.742597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43579a8f-a4de-4fc2-8dab-2939b86b7d0d None admin] PATH_INFO: `/v3/users/c6cfbce3a2d5424dabdacc0d813da0a5` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:43.762803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43579a8f-a4de-4fc2-8dab-2939b86b7d0d None admin] RBAC: Authorizing `identity:get_user(user_id=c6cfbce3a2d5424dabdacc0d813da0a5)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:43.763600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43579a8f-a4de-4fc2-8dab-2939b86b7d0d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:43.769773 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 514/1050] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:00:43 2022] GET /identity/v3/users/c6cfbce3a2d5424dabdacc0d813da0a5 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:43.775582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef4de-19dd-4bb1-a481-2d2122bb5d55 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:43.879408 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef4de-19dd-4bb1-a481-2d2122bb5d55 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:43.912294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e57dc-16a1-48ae-a8ed-f413a78e5092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:43.913002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e57dc-16a1-48ae-a8ed-f413a78e5092 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:43.913282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e57dc-16a1-48ae-a8ed-f413a78e5092 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:43.913591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e57dc-16a1-48ae-a8ed-f413a78e5092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:43.957084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:43.983518 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e57dc-16a1-48ae-a8ed-f413a78e5092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:43.984975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e57dc-16a1-48ae-a8ed-f413a78e5092 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:44.085502 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 537/1051] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:00:44.091366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43579a8f-a4de-4fc2-8dab-2939b86b7d0d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:44.100565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cebd5-c301-4ef3-81e0-d3c752e10664 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:44.100852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cebd5-c301-4ef3-81e0-d3c752e10664 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:44.101129 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cebd5-c301-4ef3-81e0-d3c752e10664 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:44.102718 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 538/1052] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:44.107407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3698c-bc47-403f-83d1-671839ff3f3e None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:44.107668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3698c-bc47-403f-83d1-671839ff3f3e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:44.107938 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3698c-bc47-403f-83d1-671839ff3f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:44.124885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-cef3698c-bc47-403f-83d1-671839ff3f3e None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:44.126056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-cef3698c-bc47-403f-83d1-671839ff3f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:44.192074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43579a8f-a4de-4fc2-8dab-2939b86b7d0d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:44.193126 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 539/1053] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:44.216343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3698c-bc47-403f-83d1-671839ff3f3e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:44.219496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:44.224632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:44.234486 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd84d56-add2-4f98-8ea2-d207ee2ffd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:44.235369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd84d56-add2-4f98-8ea2-d207ee2ffd28 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:44.235651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd84d56-add2-4f98-8ea2-d207ee2ffd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:44.235926 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd84d56-add2-4f98-8ea2-d207ee2ffd28 None admin] PATH_INFO: `/v3/projects/6b9b4da5eeb24eb39a5400029860a693` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:44.237390 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd84d56-add2-4f98-8ea2-d207ee2ffd28 None admin] RBAC: Authorizing `identity:get_project(project_id=6b9b4da5eeb24eb39a5400029860a693)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:44.238888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd84d56-add2-4f98-8ea2-d207ee2ffd28 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:44.245803 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 515/1054] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:00:44 2022] GET /identity/v3/projects/6b9b4da5eeb24eb39a5400029860a693 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:44.247016 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd84d56-add2-4f98-8ea2-d207ee2ffd28 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:44.317529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3698c-bc47-403f-83d1-671839ff3f3e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:44.352137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd84d56-add2-4f98-8ea2-d207ee2ffd28 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:44.358130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c1a74-3918-4664-a679-d93766a6ca1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:44.358961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c1a74-3918-4664-a679-d93766a6ca1c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:44.359464 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c1a74-3918-4664-a679-d93766a6ca1c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:44.359660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c1a74-3918-4664-a679-d93766a6ca1c None admin] PATH_INFO: `/v3/services/image` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:44.360260 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c1a74-3918-4664-a679-d93766a6ca1c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:44.361734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c1a74-3918-4664-a679-d93766a6ca1c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-477c1a74-3918-4664-a679-d93766a6ca1c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:44.373747 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:44.374824 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:44.375698 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 540/1055] 10.222.0.16 () {56 vars in 1105 bytes} [Tue Jul 5 10:00:44 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:44.381051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c1a74-3918-4664-a679-d93766a6ca1c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:44.390357 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c145b-5ca5-4116-bd65-4442d6ac6d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:44.391018 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c145b-5ca5-4116-bd65-4442d6ac6d97 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:44.391303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c145b-5ca5-4116-bd65-4442d6ac6d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:44.391565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c145b-5ca5-4116-bd65-4442d6ac6d97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:44.392413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c145b-5ca5-4116-bd65-4442d6ac6d97 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=6b9b4da5eeb24eb39a5400029860a693, user.id=c6cfbce3a2d5424dabdacc0d813da0a5) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:44.392752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c145b-5ca5-4116-bd65-4442d6ac6d97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:44.394276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c145b-5ca5-4116-bd65-4442d6ac6d97 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:44.402394 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 516/1056] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:44 2022] GET /identity/v3/role_assignments?user.id=c6cfbce3a2d5424dabdacc0d813da0a5&scope.project.id=6b9b4da5eeb24eb39a5400029860a693&role.id=c184a177010144429f495f3290d28221 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:44.464840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:44.507916 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c1a74-3918-4664-a679-d93766a6ca1c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:44.544929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60612f18-7267-44d8-80ac-2103a7a63fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:44.545608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60612f18-7267-44d8-80ac-2103a7a63fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:44.545887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60612f18-7267-44d8-80ac-2103a7a63fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:44.546161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60612f18-7267-44d8-80ac-2103a7a63fe1 None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:44.546896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60612f18-7267-44d8-80ac-2103a7a63fe1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:44.547200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60612f18-7267-44d8-80ac-2103a7a63fe1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:44.548581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60612f18-7267-44d8-80ac-2103a7a63fe1 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:44.556913 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 541/1057] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:44.562438 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c145b-5ca5-4116-bd65-4442d6ac6d97 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:44.624907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:44.665523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c145b-5ca5-4116-bd65-4442d6ac6d97 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:44.704936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880340ec-3dd7-4ecc-a296-e4a6bc8dd3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:44.704936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880340ec-3dd7-4ecc-a296-e4a6bc8dd3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:44.704936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880340ec-3dd7-4ecc-a296-e4a6bc8dd3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:44.704936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880340ec-3dd7-4ecc-a296-e4a6bc8dd3a0 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:44.705843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880340ec-3dd7-4ecc-a296-e4a6bc8dd3a0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:44.705843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880340ec-3dd7-4ecc-a296-e4a6bc8dd3a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:44.707341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880340ec-3dd7-4ecc-a296-e4a6bc8dd3a0 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:44.717083 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 517/1058] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:44.718912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60612f18-7267-44d8-80ac-2103a7a63fe1 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:44.832554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60612f18-7267-44d8-80ac-2103a7a63fe1 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:44.863420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:44.869930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087732c8-939e-49ff-b7b3-e3c91dd66d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:44.870639 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087732c8-939e-49ff-b7b3-e3c91dd66d21 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:44.870922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087732c8-939e-49ff-b7b3-e3c91dd66d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:44.871203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087732c8-939e-49ff-b7b3-e3c91dd66d21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:44.871947 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087732c8-939e-49ff-b7b3-e3c91dd66d21 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:44.873318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087732c8-939e-49ff-b7b3-e3c91dd66d21 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:44.894541 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 542/1059] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jul 5 10:00:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:00:45.597420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a56a3-d765-40b9-909c-733cdcab2eca None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:45.597853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a56a3-d765-40b9-909c-733cdcab2eca None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:45.597977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a56a3-d765-40b9-909c-733cdcab2eca None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:45.599615 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 518/1060] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:45.604790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa834-a82d-4d1b-bc71-eb40b75fc3be None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:45.605151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa834-a82d-4d1b-bc71-eb40b75fc3be None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:45.605501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa834-a82d-4d1b-bc71-eb40b75fc3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:45.623171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-dd1fa834-a82d-4d1b-bc71-eb40b75fc3be None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:45.624372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-dd1fa834-a82d-4d1b-bc71-eb40b75fc3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:45.700733 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 543/1061] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:45.723437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a56a3-d765-40b9-909c-733cdcab2eca None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:45.835668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a56a3-d765-40b9-909c-733cdcab2eca None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:45.863161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:45.872805 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c5346-050f-46df-b99c-72c12739c0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:45.873655 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c5346-050f-46df-b99c-72c12739c0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:45.873940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c5346-050f-46df-b99c-72c12739c0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:45.874213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c5346-050f-46df-b99c-72c12739c0d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:45.884532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c5346-050f-46df-b99c-72c12739c0d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-5f8c5346-050f-46df-b99c-72c12739c0d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:45.892793 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:45.893738 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:45.894671 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 519/1062] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jul 5 10:00:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:45.901787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fa834-a82d-4d1b-bc71-eb40b75fc3be None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:46.007842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fa834-a82d-4d1b-bc71-eb40b75fc3be None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:46.022839 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:46.045511 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9851675-ce2a-4522-a259-a8a0cdabbcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:46.046194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9851675-ce2a-4522-a259-a8a0cdabbcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:46.046506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9851675-ce2a-4522-a259-a8a0cdabbcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:46.046783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9851675-ce2a-4522-a259-a8a0cdabbcd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:46.047537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9851675-ce2a-4522-a259-a8a0cdabbcd5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:46.047838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9851675-ce2a-4522-a259-a8a0cdabbcd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:46.049249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9851675-ce2a-4522-a259-a8a0cdabbcd5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:46.057544 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 544/1063] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jul 5 10:00:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:46.063249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c5346-050f-46df-b99c-72c12739c0d5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:46.167636 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c5346-050f-46df-b99c-72c12739c0d5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:46.200468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2117392-3c83-44f1-b721-ed055df4c25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:46.201297 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2117392-3c83-44f1-b721-ed055df4c25b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:46.201594 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2117392-3c83-44f1-b721-ed055df4c25b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:46.201869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2117392-3c83-44f1-b721-ed055df4c25b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:46.212818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2117392-3c83-44f1-b721-ed055df4c25b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:46.214302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2117392-3c83-44f1-b721-ed055df4c25b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-f2117392-3c83-44f1-b721-ed055df4c25b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:46.231286 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:46.232188 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:46.238128 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:46.238128 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:46.238128 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:46.238128 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:46.238128 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 520/1064] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:46.239944 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9851675-ce2a-4522-a259-a8a0cdabbcd5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:46.348410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9851675-ce2a-4522-a259-a8a0cdabbcd5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:46.383381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ecc96-2050-4a8f-a5b3-2774972d0644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:46.384066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ecc96-2050-4a8f-a5b3-2774972d0644 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:46.384343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ecc96-2050-4a8f-a5b3-2774972d0644 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:46.384614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ecc96-2050-4a8f-a5b3-2774972d0644 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:46.385335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ecc96-2050-4a8f-a5b3-2774972d0644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:46.385644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ecc96-2050-4a8f-a5b3-2774972d0644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:46.387070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ecc96-2050-4a8f-a5b3-2774972d0644 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:46.398728 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de2d3a-e97c-4c71-af3c-7c9e6737323b None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:46.399006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de2d3a-e97c-4c71-af3c-7c9e6737323b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:46.399268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de2d3a-e97c-4c71-af3c-7c9e6737323b None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:46.404009 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 545/1065] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:46.404231 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 521/1066] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:46.405696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de2d3a-e97c-4c71-af3c-7c9e6737323b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:46.405921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa3595-e4c9-4924-b6f8-ed27802a1e29 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:46.406184 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa3595-e4c9-4924-b6f8-ed27802a1e29 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:46.406493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa3595-e4c9-4924-b6f8-ed27802a1e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:46.423359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-48aa3595-e4c9-4924-b6f8-ed27802a1e29 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:46.424546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-48aa3595-e4c9-4924-b6f8-ed27802a1e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:46.454053 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:46.463292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:46.505590 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 546/1067] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:46.513922 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de2d3a-e97c-4c71-af3c-7c9e6737323b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:46.529799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa3595-e4c9-4924-b6f8-ed27802a1e29 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:46.550704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ceda6d-b1ff-469e-8364-dea40312c144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:46.551524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceda6d-b1ff-469e-8364-dea40312c144 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:46.551807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceda6d-b1ff-469e-8364-dea40312c144 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:46.552086 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceda6d-b1ff-469e-8364-dea40312c144 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:46.559867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ceda6d-b1ff-469e-8364-dea40312c144 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:46.561309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ceda6d-b1ff-469e-8364-dea40312c144 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d0ceda6d-b1ff-469e-8364-dea40312c144 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:46.577798 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:46.578869 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:46.579860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:46.579860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:46.579860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:46.579860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:46.579860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:46.579860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:46.579860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:46.579860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:46.579860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:46.579860 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:46.579860 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 522/1068] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jul 5 10:00:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:46.587232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ceda6d-b1ff-469e-8364-dea40312c144 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:46.640591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa3595-e4c9-4924-b6f8-ed27802a1e29 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:46.678144 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0df39a-7db1-4158-9de9-fe69b8f5a7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:46.678895 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0df39a-7db1-4158-9de9-fe69b8f5a7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:46.679180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0df39a-7db1-4158-9de9-fe69b8f5a7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:46.679453 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0df39a-7db1-4158-9de9-fe69b8f5a7f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:46.684001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:46.686981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ceda6d-b1ff-469e-8364-dea40312c144 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:46.690951 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0df39a-7db1-4158-9de9-fe69b8f5a7f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-fb0df39a-7db1-4158-9de9-fe69b8f5a7f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:46.698972 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:46.699973 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:46.700930 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:46.705529 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 547/1069] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:46.706761 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0df39a-7db1-4158-9de9-fe69b8f5a7f0 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:46.729581 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691174d-6b9d-4dd8-a08f-01c2387ef6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:46.730711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691174d-6b9d-4dd8-a08f-01c2387ef6ca None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:46.731002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691174d-6b9d-4dd8-a08f-01c2387ef6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:46.731427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691174d-6b9d-4dd8-a08f-01c2387ef6ca None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:46.732601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691174d-6b9d-4dd8-a08f-01c2387ef6ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:46.733057 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691174d-6b9d-4dd8-a08f-01c2387ef6ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:46.734318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:46.734528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691174d-6b9d-4dd8-a08f-01c2387ef6ca None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:46.754480 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 523/1070] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jul 5 10:00:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:46.760020 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691174d-6b9d-4dd8-a08f-01c2387ef6ca None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:46.823025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0df39a-7db1-4158-9de9-fe69b8f5a7f0 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:46.861046 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7eb01-732b-4043-89e9-724bde7ada9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:46.861751 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7eb01-732b-4043-89e9-724bde7ada9a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:46.862017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7eb01-732b-4043-89e9-724bde7ada9a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:46.862287 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7eb01-732b-4043-89e9-724bde7ada9a None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:46.863032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7eb01-732b-4043-89e9-724bde7ada9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:46.863371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7eb01-732b-4043-89e9-724bde7ada9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:46.864749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7eb01-732b-4043-89e9-724bde7ada9a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:46.872890 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691174d-6b9d-4dd8-a08f-01c2387ef6ca None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:46.879065 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 548/1071] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:00:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:46.880122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7eb01-732b-4043-89e9-724bde7ada9a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:46.916706 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bc69e-944a-4300-bdc4-517fe30091d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:46.917752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bc69e-944a-4300-bdc4-517fe30091d9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:46.918182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bc69e-944a-4300-bdc4-517fe30091d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:46.918627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bc69e-944a-4300-bdc4-517fe30091d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:46.931853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bc69e-944a-4300-bdc4-517fe30091d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:46.933287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bc69e-944a-4300-bdc4-517fe30091d9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d76bc69e-944a-4300-bdc4-517fe30091d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:46.950775 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:46.951863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:46.952815 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:46.952815 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:46.952815 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:46.952815 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:46.952815 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 524/1072] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:46.953515 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7bf28-c2f6-486c-8cf0-dc022b17175e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:46.953799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7bf28-c2f6-486c-8cf0-dc022b17175e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:46.954073 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7bf28-c2f6-486c-8cf0-dc022b17175e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:46.955417 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 525/1073] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:46.958624 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7bf28-c2f6-486c-8cf0-dc022b17175e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:46.968289 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:46.970582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:46.980896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7eb01-732b-4043-89e9-724bde7ada9a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:47.013137 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28603b89-b1f9-4fb2-8bb4-c6370ea0491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:47.014011 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28603b89-b1f9-4fb2-8bb4-c6370ea0491f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:47.014311 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28603b89-b1f9-4fb2-8bb4-c6370ea0491f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:47.014603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28603b89-b1f9-4fb2-8bb4-c6370ea0491f None admin] PATH_INFO: `/v3/users/c6cfbce3a2d5424dabdacc0d813da0a5` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:47.015759 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28603b89-b1f9-4fb2-8bb4-c6370ea0491f None admin] RBAC: Authorizing `identity:get_user(user_id=c6cfbce3a2d5424dabdacc0d813da0a5)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:47.017220 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28603b89-b1f9-4fb2-8bb4-c6370ea0491f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:47.018990 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 549/1074] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:00:46 2022] GET /identity/v3/users/c6cfbce3a2d5424dabdacc0d813da0a5 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:47.020106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4405915-0baa-4028-bb53-16d21c708bcf None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:47.020386 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4405915-0baa-4028-bb53-16d21c708bcf None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:47.020660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4405915-0baa-4028-bb53-16d21c708bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:47.036701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e4405915-0baa-4028-bb53-16d21c708bcf None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:47.037902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e4405915-0baa-4028-bb53-16d21c708bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:47.059922 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7bf28-c2f6-486c-8cf0-dc022b17175e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:47.093500 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4ffaa-2cb2-4e77-87a0-7c4600ecc560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:47.094309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4ffaa-2cb2-4e77-87a0-7c4600ecc560 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:47.094604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4ffaa-2cb2-4e77-87a0-7c4600ecc560 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:47.094885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4ffaa-2cb2-4e77-87a0-7c4600ecc560 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:47.095648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4ffaa-2cb2-4e77-87a0-7c4600ecc560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:47.095960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4ffaa-2cb2-4e77-87a0-7c4600ecc560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:47.097427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4ffaa-2cb2-4e77-87a0-7c4600ecc560 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:47.101603 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 550/1075] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:47.102270 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4405915-0baa-4028-bb53-16d21c708bcf None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:47.120733 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 526/1076] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:47.121820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4ffaa-2cb2-4e77-87a0-7c4600ecc560 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:47.186436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:47.197217 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:47.206963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4405915-0baa-4028-bb53-16d21c708bcf None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:47.231112 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4ffaa-2cb2-4e77-87a0-7c4600ecc560 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:47.245790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f01018-6395-4533-a310-c96ba3769700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:47.246714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f01018-6395-4533-a310-c96ba3769700 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:47.247003 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f01018-6395-4533-a310-c96ba3769700 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:47.247284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f01018-6395-4533-a310-c96ba3769700 None admin] PATH_INFO: `/v3/projects/6b9b4da5eeb24eb39a5400029860a693` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:47.248530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f01018-6395-4533-a310-c96ba3769700 None admin] RBAC: Authorizing `identity:get_project(project_id=6b9b4da5eeb24eb39a5400029860a693)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:47.249980 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f01018-6395-4533-a310-c96ba3769700 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:47.251742 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 551/1077] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:00:47 2022] GET /identity/v3/projects/6b9b4da5eeb24eb39a5400029860a693 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:47.252491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f01018-6395-4533-a310-c96ba3769700 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:47.271053 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c93cce-f0cd-4f30-8b57-526805dc503b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:47.271853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c93cce-f0cd-4f30-8b57-526805dc503b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:47.272131 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c93cce-f0cd-4f30-8b57-526805dc503b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:47.272404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c93cce-f0cd-4f30-8b57-526805dc503b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:47.279018 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c93cce-f0cd-4f30-8b57-526805dc503b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:47.280452 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c93cce-f0cd-4f30-8b57-526805dc503b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-78c93cce-f0cd-4f30-8b57-526805dc503b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:47.295125 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:47.296100 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:47.297027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:47.297027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:47.297027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:47.297027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:47.297027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:47.297027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:47.297027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:47.297027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:47.297027 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 527/1078] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:47.297027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c93cce-f0cd-4f30-8b57-526805dc503b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:47.357334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f01018-6395-4533-a310-c96ba3769700 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:47.391790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0eb428-51a9-42e6-a070-311700820add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:47.392447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eb428-51a9-42e6-a070-311700820add None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:47.392727 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eb428-51a9-42e6-a070-311700820add None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:47.392999 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eb428-51a9-42e6-a070-311700820add None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:47.401324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c93cce-f0cd-4f30-8b57-526805dc503b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:47.416659 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:47.429691 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:47.435614 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a3764-274c-4b3f-9865-ff5e8eca081a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:47.436575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a3764-274c-4b3f-9865-ff5e8eca081a None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:47.436827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a3764-274c-4b3f-9865-ff5e8eca081a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:47.437091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a3764-274c-4b3f-9865-ff5e8eca081a None admin] PATH_INFO: `/v3/projects/6b9b4da5eeb24eb39a5400029860a693/users/c6cfbce3a2d5424dabdacc0d813da0a5/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:47.439049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a3764-274c-4b3f-9865-ff5e8eca081a None admin] RBAC: Authorizing `identity:create_grant(project_id=6b9b4da5eeb24eb39a5400029860a693, user_id=c6cfbce3a2d5424dabdacc0d813da0a5, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:47.440545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a3764-274c-4b3f-9865-ff5e8eca081a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:47.453508 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 528/1079] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 10:00:47 2022] PUT /identity/v3/projects/6b9b4da5eeb24eb39a5400029860a693/users/c6cfbce3a2d5424dabdacc0d813da0a5/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:00:47.454313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a3764-274c-4b3f-9865-ff5e8eca081a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:47.462290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0eb428-51a9-42e6-a070-311700820add None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:47.463781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0eb428-51a9-42e6-a070-311700820add None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:47.569360 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a3764-274c-4b3f-9865-ff5e8eca081a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:47.575441 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 552/1080] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:00:47.581820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0eb428-51a9-42e6-a070-311700820add None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:47.618174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a7461-dca9-464b-84b6-c8206b3090ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:47.618904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a7461-dca9-464b-84b6-c8206b3090ab None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:47.619191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a7461-dca9-464b-84b6-c8206b3090ab None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:47.619468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a7461-dca9-464b-84b6-c8206b3090ab None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:47.620848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a7461-dca9-464b-84b6-c8206b3090ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:47.620848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a7461-dca9-464b-84b6-c8206b3090ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:47.624853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a7461-dca9-464b-84b6-c8206b3090ab None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:47.636317 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 529/1081] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:47.642036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a7461-dca9-464b-84b6-c8206b3090ab None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:47.645703 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:47.692809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0eb428-51a9-42e6-a070-311700820add None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:47.695201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:47.729034 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17325add-768d-48ed-b4d9-0316e3bf503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:47.729848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17325add-768d-48ed-b4d9-0316e3bf503a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:47.730179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17325add-768d-48ed-b4d9-0316e3bf503a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:47.730491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17325add-768d-48ed-b4d9-0316e3bf503a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:47.740816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17325add-768d-48ed-b4d9-0316e3bf503a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:47.742300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17325add-768d-48ed-b4d9-0316e3bf503a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:47.747835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a7461-dca9-464b-84b6-c8206b3090ab None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-17325add-768d-48ed-b4d9-0316e3bf503a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:47.760400 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:47.761354 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:47.762382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:47.762382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:47.762382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:47.762382 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:47.762382 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 553/1082] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:47.767375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17325add-768d-48ed-b4d9-0316e3bf503a None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:47.788043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505da2c0-f3e9-4c97-a809-728de4ef3918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:47.788950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505da2c0-f3e9-4c97-a809-728de4ef3918 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:47.789648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505da2c0-f3e9-4c97-a809-728de4ef3918 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:47.789929 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505da2c0-f3e9-4c97-a809-728de4ef3918 None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/fee6ee8f99c34aa281bf02ecd3f23216/roles/9f8e96f6abc44873a5d208d80eb06f0c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:47.792906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505da2c0-f3e9-4c97-a809-728de4ef3918 None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=fee6ee8f99c34aa281bf02ecd3f23216, role_id=9f8e96f6abc44873a5d208d80eb06f0c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:47.794793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505da2c0-f3e9-4c97-a809-728de4ef3918 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:47.810547 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 530/1083] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 10:00:47 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/fee6ee8f99c34aa281bf02ecd3f23216/roles/9f8e96f6abc44873a5d208d80eb06f0c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:00:47.881317 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17325add-768d-48ed-b4d9-0316e3bf503a None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:47.889990 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:47.920896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d809568-c446-4400-8a6a-a95a7dcba289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:47.921597 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d809568-c446-4400-8a6a-a95a7dcba289 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:47.921878 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d809568-c446-4400-8a6a-a95a7dcba289 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:47.922159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d809568-c446-4400-8a6a-a95a7dcba289 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:47.922937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d809568-c446-4400-8a6a-a95a7dcba289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:47.923279 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d809568-c446-4400-8a6a-a95a7dcba289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:47.924750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d809568-c446-4400-8a6a-a95a7dcba289 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:47.939020 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 554/1084] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:49.486044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d30ee-847c-4e64-8466-d0ec327cc606 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:49.486657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d30ee-847c-4e64-8466-d0ec327cc606 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:49.486657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d30ee-847c-4e64-8466-d0ec327cc606 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:49.487996 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 531/1085] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:49.493362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b32dd-f50d-4988-8236-1a49804067fd None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:49.493799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b32dd-f50d-4988-8236-1a49804067fd None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:49.493888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b32dd-f50d-4988-8236-1a49804067fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:49.511236 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-108b32dd-f50d-4988-8236-1a49804067fd None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:49.512416 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-108b32dd-f50d-4988-8236-1a49804067fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:49.580245 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 555/1086] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:49.604424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d30ee-847c-4e64-8466-d0ec327cc606 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:49.708787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d30ee-847c-4e64-8466-d0ec327cc606 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:49.719812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:49.744305 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c6e4a-af5b-4ce3-9d2e-6c7082761b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:49.745125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c6e4a-af5b-4ce3-9d2e-6c7082761b57 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:49.745409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c6e4a-af5b-4ce3-9d2e-6c7082761b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:49.745682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c6e4a-af5b-4ce3-9d2e-6c7082761b57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:49.759591 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c6e4a-af5b-4ce3-9d2e-6c7082761b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-045c6e4a-af5b-4ce3-9d2e-6c7082761b57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:49.767217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:49.768245 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:49.769138 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 532/1087] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:00:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:49.774586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b32dd-f50d-4988-8236-1a49804067fd None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:49.809415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc2385-aeec-419f-aa3a-0c973586e76d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:49.809680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc2385-aeec-419f-aa3a-0c973586e76d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:49.809982 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc2385-aeec-419f-aa3a-0c973586e76d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:49.811558 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 533/1088] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:00:49.816448 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16049bf-2fb4-41eb-be43-e39701d2015d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:49.816694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16049bf-2fb4-41eb-be43-e39701d2015d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:49.817001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16049bf-2fb4-41eb-be43-e39701d2015d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:49.835477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e16049bf-2fb4-41eb-be43-e39701d2015d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:49.836667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e16049bf-2fb4-41eb-be43-e39701d2015d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:49.876927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:49.883155 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b32dd-f50d-4988-8236-1a49804067fd None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:49.903202 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 534/1089] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:00:49.916115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6153a5-7cbc-49b0-bb14-385d3a564101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:49.916785 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6153a5-7cbc-49b0-bb14-385d3a564101 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:49.917065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6153a5-7cbc-49b0-bb14-385d3a564101 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:49.917336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6153a5-7cbc-49b0-bb14-385d3a564101 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:49.918077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6153a5-7cbc-49b0-bb14-385d3a564101 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:49.918413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6153a5-7cbc-49b0-bb14-385d3a564101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:49.919775 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6153a5-7cbc-49b0-bb14-385d3a564101 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:49.926185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16049bf-2fb4-41eb-be43-e39701d2015d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:49.934123 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 556/1090] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:00:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:49.935540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6153a5-7cbc-49b0-bb14-385d3a564101 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:50.025416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:50.029375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16049bf-2fb4-41eb-be43-e39701d2015d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:50.030856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6153a5-7cbc-49b0-bb14-385d3a564101 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:50.062689 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb239f-f108-47ae-a700-02e46a07fa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:50.063569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb239f-f108-47ae-a700-02e46a07fa20 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:50.063847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb239f-f108-47ae-a700-02e46a07fa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:50.064148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb239f-f108-47ae-a700-02e46a07fa20 None admin] PATH_INFO: `/v3/users/c6cfbce3a2d5424dabdacc0d813da0a5` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:50.064461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d2faa-0dad-4a4d-a249-7edae47bcdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:50.065307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb239f-f108-47ae-a700-02e46a07fa20 None admin] RBAC: Authorizing `identity:get_user(user_id=c6cfbce3a2d5424dabdacc0d813da0a5)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:50.065409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2faa-0dad-4a4d-a249-7edae47bcdef None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:50.065586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2faa-0dad-4a4d-a249-7edae47bcdef None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:50.065857 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2faa-0dad-4a4d-a249-7edae47bcdef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:50.066776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb239f-f108-47ae-a700-02e46a07fa20 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:50.068399 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 557/1091] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:00:49 2022] GET /identity/v3/users/c6cfbce3a2d5424dabdacc0d813da0a5 => generated 285 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:50.073957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb239f-f108-47ae-a700-02e46a07fa20 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:50.076280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d2faa-0dad-4a4d-a249-7edae47bcdef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ac2d2faa-0dad-4a4d-a249-7edae47bcdef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:00:50.085227 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:50.086196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:50.087073 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 535/1092] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jul 5 10:00:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:50.092059 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d2faa-0dad-4a4d-a249-7edae47bcdef None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:50.106264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:50.179206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb239f-f108-47ae-a700-02e46a07fa20 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:50.194181 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d2faa-0dad-4a4d-a249-7edae47bcdef None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:50.214644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d16d6-df2e-4a98-ba50-8fcf02538757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:50.215295 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d16d6-df2e-4a98-ba50-8fcf02538757 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:50.215561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d16d6-df2e-4a98-ba50-8fcf02538757 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:50.215829 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d16d6-df2e-4a98-ba50-8fcf02538757 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:50.229022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297f359-5dc6-4841-9734-0cd56a8b0724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:50.229721 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297f359-5dc6-4841-9734-0cd56a8b0724 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:50.230038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297f359-5dc6-4841-9734-0cd56a8b0724 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:50.230365 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297f359-5dc6-4841-9734-0cd56a8b0724 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:50.231143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297f359-5dc6-4841-9734-0cd56a8b0724 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:50.231476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297f359-5dc6-4841-9734-0cd56a8b0724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:50.232969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297f359-5dc6-4841-9734-0cd56a8b0724 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:50.241749 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 536/1093] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jul 5 10:00:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:50.247539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297f359-5dc6-4841-9734-0cd56a8b0724 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:50.272489 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:50.282701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d16d6-df2e-4a98-ba50-8fcf02538757 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:50.284092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d16d6-df2e-4a98-ba50-8fcf02538757 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:50.309745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:50.349409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297f359-5dc6-4841-9734-0cd56a8b0724 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:50.380708 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 558/1094] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 10:00:50.384400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376584f-0da5-4a64-8cf4-c6de0eae9afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:50.385208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376584f-0da5-4a64-8cf4-c6de0eae9afc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:50.385511 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376584f-0da5-4a64-8cf4-c6de0eae9afc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:50.385850 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376584f-0da5-4a64-8cf4-c6de0eae9afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:50.386661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d16d6-df2e-4a98-ba50-8fcf02538757 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:50.396962 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376584f-0da5-4a64-8cf4-c6de0eae9afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:50.398432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376584f-0da5-4a64-8cf4-c6de0eae9afc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-c376584f-0da5-4a64-8cf4-c6de0eae9afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:50.415222 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:50.416183 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:50.417216 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:50.417216 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:50.417216 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:50.417216 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:50.417216 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 537/1095] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:50.422006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376584f-0da5-4a64-8cf4-c6de0eae9afc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:50.482557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d16d6-df2e-4a98-ba50-8fcf02538757 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:50.493998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:50.507653 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:50.517594 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8ec28-3338-4b43-9c17-9ff3f840e149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:50.518558 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8ec28-3338-4b43-9c17-9ff3f840e149 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:50.518891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8ec28-3338-4b43-9c17-9ff3f840e149 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:50.519173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8ec28-3338-4b43-9c17-9ff3f840e149 None admin] PATH_INFO: `/v3/projects/6b9b4da5eeb24eb39a5400029860a693` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:50.519575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376584f-0da5-4a64-8cf4-c6de0eae9afc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:50.520390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a8ec28-3338-4b43-9c17-9ff3f840e149 None admin] RBAC: Authorizing `identity:get_project(project_id=6b9b4da5eeb24eb39a5400029860a693)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:50.521858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a8ec28-3338-4b43-9c17-9ff3f840e149 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:50.523592 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 559/1096] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:00:50 2022] GET /identity/v3/projects/6b9b4da5eeb24eb39a5400029860a693 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:50.529523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8ec28-3338-4b43-9c17-9ff3f840e149 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:50.554665 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102067d-5fb0-4ab1-b668-62adace9abcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:50.555261 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102067d-5fb0-4ab1-b668-62adace9abcf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:50.555545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102067d-5fb0-4ab1-b668-62adace9abcf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:50.555819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102067d-5fb0-4ab1-b668-62adace9abcf None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:50.556550 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102067d-5fb0-4ab1-b668-62adace9abcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:50.556859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102067d-5fb0-4ab1-b668-62adace9abcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:50.558251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102067d-5fb0-4ab1-b668-62adace9abcf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:50.569785 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 538/1097] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:50.575398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102067d-5fb0-4ab1-b668-62adace9abcf None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:50.626475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8ec28-3338-4b43-9c17-9ff3f840e149 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:50.660656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60c245-6c23-4e09-9293-4b1df0d9f407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:50.661309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60c245-6c23-4e09-9293-4b1df0d9f407 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:50.661591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60c245-6c23-4e09-9293-4b1df0d9f407 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:50.661868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60c245-6c23-4e09-9293-4b1df0d9f407 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:50.662728 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60c245-6c23-4e09-9293-4b1df0d9f407 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=6b9b4da5eeb24eb39a5400029860a693, user.id=c6cfbce3a2d5424dabdacc0d813da0a5) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:50.663035 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60c245-6c23-4e09-9293-4b1df0d9f407 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:50.664476 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60c245-6c23-4e09-9293-4b1df0d9f407 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:50.686251 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 560/1098] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:50 2022] GET /identity/v3/role_assignments?user.id=c6cfbce3a2d5424dabdacc0d813da0a5&scope.project.id=6b9b4da5eeb24eb39a5400029860a693&role.id=c184a177010144429f495f3290d28221 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:50.686485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102067d-5fb0-4ab1-b668-62adace9abcf None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:50.710736 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809af332-ec3a-4284-9cfc-823a1c21c551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:50.711550 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809af332-ec3a-4284-9cfc-823a1c21c551 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:50.711833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809af332-ec3a-4284-9cfc-823a1c21c551 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:50.712110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809af332-ec3a-4284-9cfc-823a1c21c551 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:50.714690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:50.721217 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809af332-ec3a-4284-9cfc-823a1c21c551 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:50.722835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809af332-ec3a-4284-9cfc-823a1c21c551 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-809af332-ec3a-4284-9cfc-823a1c21c551 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:00:50.740122 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:50.741168 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:50.742109 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:50.742109 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:50.742109 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:00:50.742109 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:00:50.742109 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:00:50.742109 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:00:50.742109 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:00:50.742109 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:00:50.742109 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 10:00:50.742109 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:00:50.742109 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 539/1099] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jul 5 10:00:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:00:50.747694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60c245-6c23-4e09-9293-4b1df0d9f407 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:50.812902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:50.856859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60c245-6c23-4e09-9293-4b1df0d9f407 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:50.889563 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263ea54-a89a-4c64-a9f9-5e7d4c55c9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:50.890247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263ea54-a89a-4c64-a9f9-5e7d4c55c9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:50.890555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263ea54-a89a-4c64-a9f9-5e7d4c55c9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:50.890830 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263ea54-a89a-4c64-a9f9-5e7d4c55c9d1 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:50.891669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263ea54-a89a-4c64-a9f9-5e7d4c55c9d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:50.891979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263ea54-a89a-4c64-a9f9-5e7d4c55c9d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:50.893468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263ea54-a89a-4c64-a9f9-5e7d4c55c9d1 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:50.910132 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 561/1100] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jul 5 10:00:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:50.915907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809af332-ec3a-4284-9cfc-823a1c21c551 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:51.018336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809af332-ec3a-4284-9cfc-823a1c21c551 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:51.053491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbea7e-4def-4e85-9a2d-e00b97ab813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:51.054137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbea7e-4def-4e85-9a2d-e00b97ab813e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:51.054447 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbea7e-4def-4e85-9a2d-e00b97ab813e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:51.054724 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbea7e-4def-4e85-9a2d-e00b97ab813e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:51.115780 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:51.123793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbea7e-4def-4e85-9a2d-e00b97ab813e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:51.125299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbea7e-4def-4e85-9a2d-e00b97ab813e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:51.224947 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 540/1101] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:00:51.230965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263ea54-a89a-4c64-a9f9-5e7d4c55c9d1 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:51.332607 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263ea54-a89a-4c64-a9f9-5e7d4c55c9d1 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:51.349838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:51.368045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51468420-eca3-49a8-ab4e-e257ea741117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:51.368772 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51468420-eca3-49a8-ab4e-e257ea741117 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:51.369053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51468420-eca3-49a8-ab4e-e257ea741117 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:51.369328 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51468420-eca3-49a8-ab4e-e257ea741117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:51.379651 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51468420-eca3-49a8-ab4e-e257ea741117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:51.381115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51468420-eca3-49a8-ab4e-e257ea741117 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-51468420-eca3-49a8-ab4e-e257ea741117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:00:51.398258 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:51.399280 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:51.400213 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:00:51.400213 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:00:51.400213 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:00:51.400213 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:51.400213 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 562/1102] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:51.405302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbea7e-4def-4e85-9a2d-e00b97ab813e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:51.497689 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:51.504952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbea7e-4def-4e85-9a2d-e00b97ab813e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:51.538199 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4ffa0-c824-4a53-81fc-f24dde1ef893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:51.538901 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4ffa0-c824-4a53-81fc-f24dde1ef893 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:51.539180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4ffa0-c824-4a53-81fc-f24dde1ef893 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:51.539449 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4ffa0-c824-4a53-81fc-f24dde1ef893 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:51.540166 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4ffa0-c824-4a53-81fc-f24dde1ef893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:51.540470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4ffa0-c824-4a53-81fc-f24dde1ef893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:51.541872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4ffa0-c824-4a53-81fc-f24dde1ef893 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:51.552947 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 541/1103] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:51.558472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51468420-eca3-49a8-ab4e-e257ea741117 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:51.657210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51468420-eca3-49a8-ab4e-e257ea741117 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:51.692468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06471823-12e0-4c7b-aa9d-77dba8a5e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:51.693263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06471823-12e0-4c7b-aa9d-77dba8a5e0cd None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:51.693538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06471823-12e0-4c7b-aa9d-77dba8a5e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:51.693805 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06471823-12e0-4c7b-aa9d-77dba8a5e0cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:51.700386 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06471823-12e0-4c7b-aa9d-77dba8a5e0cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:51.701827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06471823-12e0-4c7b-aa9d-77dba8a5e0cd None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-06471823-12e0-4c7b-aa9d-77dba8a5e0cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:00:51.715598 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:00:51.716536 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:00:51.717602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:00:51.717602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:00:51.717602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:00:51.717602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:00:51.717602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:00:51.717602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:00:51.717602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:00:51.717602 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:00:51.717602 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 563/1104] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:00:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:00:51.722734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4ffa0-c824-4a53-81fc-f24dde1ef893 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:51.831176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4ffa0-c824-4a53-81fc-f24dde1ef893 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:51.864481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec5a11-dea9-4c9d-b1f0-c52a8807b79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:51.865142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec5a11-dea9-4c9d-b1f0-c52a8807b79e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:51.865427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec5a11-dea9-4c9d-b1f0-c52a8807b79e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:51.865708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec5a11-dea9-4c9d-b1f0-c52a8807b79e None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:51.866534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec5a11-dea9-4c9d-b1f0-c52a8807b79e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:51.866847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec5a11-dea9-4c9d-b1f0-c52a8807b79e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:51.868302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec5a11-dea9-4c9d-b1f0-c52a8807b79e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:51.880182 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 542/1105] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:00:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:51.886587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06471823-12e0-4c7b-aa9d-77dba8a5e0cd None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:51.916917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:51.989726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06471823-12e0-4c7b-aa9d-77dba8a5e0cd None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:52.023495 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0e077-7122-4a33-b5f8-da919d7f71e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:52.024145 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0e077-7122-4a33-b5f8-da919d7f71e3 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:52.024400 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0e077-7122-4a33-b5f8-da919d7f71e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:52.024697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0e077-7122-4a33-b5f8-da919d7f71e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:52.025525 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0e077-7122-4a33-b5f8-da919d7f71e3 None admin] RBAC: Adding query filter params (role.id=9f8e96f6abc44873a5d208d80eb06f0c, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=fee6ee8f99c34aa281bf02ecd3f23216) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:52.025837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0e077-7122-4a33-b5f8-da919d7f71e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:52.027334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0e077-7122-4a33-b5f8-da919d7f71e3 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:52.033322 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 564/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:51 2022] GET /identity/v3/role_assignments?user.id=fee6ee8f99c34aa281bf02ecd3f23216&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=9f8e96f6abc44873a5d208d80eb06f0c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:52.719795 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd3f8b-a0f0-4306-bc5d-e6dc4b816a05 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:52.720402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd3f8b-a0f0-4306-bc5d-e6dc4b816a05 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:52.720402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd3f8b-a0f0-4306-bc5d-e6dc4b816a05 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:52.722019 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 543/1107] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:52.727864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96357f6-c04e-45fb-bf8c-aef4cbb978e9 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:52.728306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96357f6-c04e-45fb-bf8c-aef4cbb978e9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:52.728417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96357f6-c04e-45fb-bf8c-aef4cbb978e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:52.747238 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b96357f6-c04e-45fb-bf8c-aef4cbb978e9 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:52.748490 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b96357f6-c04e-45fb-bf8c-aef4cbb978e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:52.827252 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 565/1108] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:52.852528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd3f8b-a0f0-4306-bc5d-e6dc4b816a05 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:52.869381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:52.968384 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd3f8b-a0f0-4306-bc5d-e6dc4b816a05 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:53.013129 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90832c-8e1c-4e2d-ad22-f8f8c3e8f472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:53.013975 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90832c-8e1c-4e2d-ad22-f8f8c3e8f472 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:53.014273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90832c-8e1c-4e2d-ad22-f8f8c3e8f472 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:53.014564 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90832c-8e1c-4e2d-ad22-f8f8c3e8f472 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:53.015886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90832c-8e1c-4e2d-ad22-f8f8c3e8f472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:53.017328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90832c-8e1c-4e2d-ad22-f8f8c3e8f472 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:53.018993 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 544/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:00:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:53.024964 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96357f6-c04e-45fb-bf8c-aef4cbb978e9 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:53.054713 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:53.144823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96357f6-c04e-45fb-bf8c-aef4cbb978e9 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:53.179758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f4436-aa6e-41b9-8167-3076fda0a32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:53.180442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f4436-aa6e-41b9-8167-3076fda0a32b None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:53.180725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f4436-aa6e-41b9-8167-3076fda0a32b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:53.180997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f4436-aa6e-41b9-8167-3076fda0a32b None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:53.181719 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f4436-aa6e-41b9-8167-3076fda0a32b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:53.183222 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f4436-aa6e-41b9-8167-3076fda0a32b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:53.203306 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 566/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 10:00:53 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:00:55.242007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2f6ed-9f0e-4052-af53-088d6948738a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:55.242921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2f6ed-9f0e-4052-af53-088d6948738a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:55.242921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2f6ed-9f0e-4052-af53-088d6948738a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:55.245154 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 545/1111] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:55.251457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda492f2-539e-4d52-a8b3-7afd7fb38b17 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:55.251904 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda492f2-539e-4d52-a8b3-7afd7fb38b17 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:55.251998 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda492f2-539e-4d52-a8b3-7afd7fb38b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:55.270577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-fda492f2-539e-4d52-a8b3-7afd7fb38b17 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:55.271815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-fda492f2-539e-4d52-a8b3-7afd7fb38b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:55.341563 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 567/1112] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:55.370941 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2f6ed-9f0e-4052-af53-088d6948738a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:55.461351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:55.482461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2f6ed-9f0e-4052-af53-088d6948738a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:55.518617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a140a6-9bce-40e0-971e-19834e4771c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:55.519373 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a140a6-9bce-40e0-971e-19834e4771c1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:55.519650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a140a6-9bce-40e0-971e-19834e4771c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:55.519920 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a140a6-9bce-40e0-971e-19834e4771c1 None admin] PATH_INFO: `/v3/roles/70d38d0a968a4e9585c5873d24cfbbb1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:55.521499 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a140a6-9bce-40e0-971e-19834e4771c1 None admin] RBAC: Authorizing `identity:get_role(role_id=70d38d0a968a4e9585c5873d24cfbbb1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:55.522918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a140a6-9bce-40e0-971e-19834e4771c1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:55.524319 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 546/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:00:55 2022] GET /identity/v3/roles/70d38d0a968a4e9585c5873d24cfbbb1 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:55.531195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda492f2-539e-4d52-a8b3-7afd7fb38b17 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:55.542455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:55.640629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda492f2-539e-4d52-a8b3-7afd7fb38b17 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:55.676528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12f20e-1faa-4c61-8f59-63c4147c1a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:55.677417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12f20e-1faa-4c61-8f59-63c4147c1a55 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:55.677704 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12f20e-1faa-4c61-8f59-63c4147c1a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:55.677972 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12f20e-1faa-4c61-8f59-63c4147c1a55 None admin] PATH_INFO: `/v3/users/739d245637684fdc97adf28ed603fb9c` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:55.691837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12f20e-1faa-4c61-8f59-63c4147c1a55 None admin] RBAC: Authorizing `identity:get_user(user_id=739d245637684fdc97adf28ed603fb9c)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:55.693274 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12f20e-1faa-4c61-8f59-63c4147c1a55 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:55.695007 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 568/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:00:55 2022] GET /identity/v3/users/739d245637684fdc97adf28ed603fb9c => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:00:55.704505 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a140a6-9bce-40e0-971e-19834e4771c1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:55.819338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a140a6-9bce-40e0-971e-19834e4771c1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:55.858961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972453b4-e850-4504-9937-144e944d69bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:55.859605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972453b4-e850-4504-9937-144e944d69bb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:55.859891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972453b4-e850-4504-9937-144e944d69bb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:55.860167 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972453b4-e850-4504-9937-144e944d69bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:55.878844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:55.932558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972453b4-e850-4504-9937-144e944d69bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:55.934007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972453b4-e850-4504-9937-144e944d69bb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:56.046943 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 547/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:00:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:00:56.053466 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12f20e-1faa-4c61-8f59-63c4147c1a55 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:56.120846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:56.156632 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12f20e-1faa-4c61-8f59-63c4147c1a55 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:56.206331 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febd457-c8ff-42c4-ae1b-76a2a19fc8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:56.207195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febd457-c8ff-42c4-ae1b-76a2a19fc8de None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:56.207486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febd457-c8ff-42c4-ae1b-76a2a19fc8de None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:56.207760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febd457-c8ff-42c4-ae1b-76a2a19fc8de None admin] PATH_INFO: `/v3/projects/6b9b4da5eeb24eb39a5400029860a693` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:56.209085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febd457-c8ff-42c4-ae1b-76a2a19fc8de None admin] RBAC: Authorizing `identity:get_project(project_id=6b9b4da5eeb24eb39a5400029860a693)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:56.210571 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febd457-c8ff-42c4-ae1b-76a2a19fc8de None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:56.212253 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 569/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:00:56 2022] GET /identity/v3/projects/6b9b4da5eeb24eb39a5400029860a693 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:56.221612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972453b4-e850-4504-9937-144e944d69bb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:56.275787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:56.332235 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972453b4-e850-4504-9937-144e944d69bb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:56.377836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e59ef-d470-468d-be80-ac1030db5672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:56.378444 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e59ef-d470-468d-be80-ac1030db5672 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:56.378728 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e59ef-d470-468d-be80-ac1030db5672 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:56.379000 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e59ef-d470-468d-be80-ac1030db5672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:56.379814 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e59ef-d470-468d-be80-ac1030db5672 None admin] RBAC: Adding query filter params (role.id=70d38d0a968a4e9585c5873d24cfbbb1, scope.project.id=6b9b4da5eeb24eb39a5400029860a693, user.id=739d245637684fdc97adf28ed603fb9c) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:00:56.380116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e59ef-d470-468d-be80-ac1030db5672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:56.381542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e59ef-d470-468d-be80-ac1030db5672 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:56.387573 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 548/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:00:56 2022] GET /identity/v3/role_assignments?user.id=739d245637684fdc97adf28ed603fb9c&scope.project.id=6b9b4da5eeb24eb39a5400029860a693&role.id=70d38d0a968a4e9585c5873d24cfbbb1 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:58.349708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b6885-3f81-4163-8f94-1b54e1c08233 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:58.350293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b6885-3f81-4163-8f94-1b54e1c08233 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:58.350293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b6885-3f81-4163-8f94-1b54e1c08233 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:58.351666 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 570/1118] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:00:58.357966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8b6f3-a6d1-456a-b92e-8061a15e5ad7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:58.359253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8b6f3-a6d1-456a-b92e-8061a15e5ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:58.359725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8b6f3-a6d1-456a-b92e-8061a15e5ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:58.384037 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-87a8b6f3-a6d1-456a-b92e-8061a15e5ad7 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:00:58.385253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-87a8b6f3-a6d1-456a-b92e-8061a15e5ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:00:58.453770 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 549/1119] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:00:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:00:58.483237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b6885-3f81-4163-8f94-1b54e1c08233 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:58.588628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b6885-3f81-4163-8f94-1b54e1c08233 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:58.623342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f4818-5365-436f-bbfa-e4561e6bf2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:58.624131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f4818-5365-436f-bbfa-e4561e6bf2eb None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:58.624413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f4818-5365-436f-bbfa-e4561e6bf2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:58.624699 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f4818-5365-436f-bbfa-e4561e6bf2eb None admin] PATH_INFO: `/v3/roles/70d38d0a968a4e9585c5873d24cfbbb1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:58.626318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f4818-5365-436f-bbfa-e4561e6bf2eb None admin] RBAC: Authorizing `identity:get_role(role_id=70d38d0a968a4e9585c5873d24cfbbb1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:58.627788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f4818-5365-436f-bbfa-e4561e6bf2eb None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:58.629244 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 571/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:00:58 2022] GET /identity/v3/roles/70d38d0a968a4e9585c5873d24cfbbb1 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:58.635815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8b6f3-a6d1-456a-b92e-8061a15e5ad7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:58.686495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:58.742104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8b6f3-a6d1-456a-b92e-8061a15e5ad7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:58.783182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f2b78-0f47-45cf-815d-30308121b3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:58.784090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f2b78-0f47-45cf-815d-30308121b3cb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:58.784371 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f2b78-0f47-45cf-815d-30308121b3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:58.784645 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f2b78-0f47-45cf-815d-30308121b3cb None admin] PATH_INFO: `/v3/users/739d245637684fdc97adf28ed603fb9c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:58.785909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f2b78-0f47-45cf-815d-30308121b3cb None admin] RBAC: Authorizing `identity:get_user(user_id=739d245637684fdc97adf28ed603fb9c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:58.787450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f2b78-0f47-45cf-815d-30308121b3cb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:58.789127 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 550/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:00:58 2022] GET /identity/v3/users/739d245637684fdc97adf28ed603fb9c => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:58.795524 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f4818-5365-436f-bbfa-e4561e6bf2eb None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:58.798777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:58.907586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f4818-5365-436f-bbfa-e4561e6bf2eb None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:58.940805 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ea563-3ce1-4d6e-ad97-88e0b6da6c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:58.941672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ea563-3ce1-4d6e-ad97-88e0b6da6c55 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:58.941961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ea563-3ce1-4d6e-ad97-88e0b6da6c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:58.942266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ea563-3ce1-4d6e-ad97-88e0b6da6c55 None admin] PATH_INFO: `/v3/projects/6b9b4da5eeb24eb39a5400029860a693` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:58.943583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ea563-3ce1-4d6e-ad97-88e0b6da6c55 None admin] RBAC: Authorizing `identity:get_project(project_id=6b9b4da5eeb24eb39a5400029860a693)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:58.945069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ea563-3ce1-4d6e-ad97-88e0b6da6c55 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:58.946842 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 572/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:00:58 2022] GET /identity/v3/projects/6b9b4da5eeb24eb39a5400029860a693 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:00:58.957087 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f2b78-0f47-45cf-815d-30308121b3cb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:00:59.061617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f2b78-0f47-45cf-815d-30308121b3cb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:00:59.074936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:00:59.103584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fb2bc-40bf-471c-861d-6ce1f323d845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:00:59.104480 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fb2bc-40bf-471c-861d-6ce1f323d845 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:00:59.104761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fb2bc-40bf-471c-861d-6ce1f323d845 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:00:59.105038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fb2bc-40bf-471c-861d-6ce1f323d845 None admin] PATH_INFO: `/v3/projects/6b9b4da5eeb24eb39a5400029860a693/users/739d245637684fdc97adf28ed603fb9c/roles/70d38d0a968a4e9585c5873d24cfbbb1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:00:59.107439 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fb2bc-40bf-471c-861d-6ce1f323d845 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b9b4da5eeb24eb39a5400029860a693, user_id=739d245637684fdc97adf28ed603fb9c, role_id=70d38d0a968a4e9585c5873d24cfbbb1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:00:59.108935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fb2bc-40bf-471c-861d-6ce1f323d845 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:00:59.127900 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 551/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 10:00:58 2022] PUT /identity/v3/projects/6b9b4da5eeb24eb39a5400029860a693/users/739d245637684fdc97adf28ed603fb9c/roles/70d38d0a968a4e9585c5873d24cfbbb1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:01:01.112802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbf7c-b74b-4bb0-b922-0c5f0eb3b48d None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:01.113744 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbf7c-b74b-4bb0-b922-0c5f0eb3b48d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:01.113744 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbf7c-b74b-4bb0-b922-0c5f0eb3b48d None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:01.115813 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 573/1124] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:01.122075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c812dd9-cede-4619-bd53-2f0af9d8c685 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:01.122853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c812dd9-cede-4619-bd53-2f0af9d8c685 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:01.123008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c812dd9-cede-4619-bd53-2f0af9d8c685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:01.147043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6c812dd9-cede-4619-bd53-2f0af9d8c685 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:01.148232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6c812dd9-cede-4619-bd53-2f0af9d8c685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:01.219549 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 552/1125] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:01.256020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfbf7c-b74b-4bb0-b922-0c5f0eb3b48d None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:01.343757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:01.379274 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfbf7c-b74b-4bb0-b922-0c5f0eb3b48d None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:01.413825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472b74-4d7b-4a95-bc50-4475575c3a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:01.414736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472b74-4d7b-4a95-bc50-4475575c3a46 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:01.415016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472b74-4d7b-4a95-bc50-4475575c3a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:01.415292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472b74-4d7b-4a95-bc50-4475575c3a46 None admin] PATH_INFO: `/v3/roles/70d38d0a968a4e9585c5873d24cfbbb1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:01.416901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48472b74-4d7b-4a95-bc50-4475575c3a46 None admin] RBAC: Authorizing `identity:get_role(role_id=70d38d0a968a4e9585c5873d24cfbbb1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:01.418316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48472b74-4d7b-4a95-bc50-4475575c3a46 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:01.419708 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 574/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:01:01 2022] GET /identity/v3/roles/70d38d0a968a4e9585c5873d24cfbbb1 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:01.426580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c812dd9-cede-4619-bd53-2f0af9d8c685 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:01.541047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c812dd9-cede-4619-bd53-2f0af9d8c685 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:01.542639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:01.575345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507408de-1e0e-4b5a-af51-1e444d12cabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:01.576207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507408de-1e0e-4b5a-af51-1e444d12cabd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:01.576488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507408de-1e0e-4b5a-af51-1e444d12cabd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:01.576761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507408de-1e0e-4b5a-af51-1e444d12cabd None admin] PATH_INFO: `/v3/users/739d245637684fdc97adf28ed603fb9c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:01.577946 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507408de-1e0e-4b5a-af51-1e444d12cabd None admin] RBAC: Authorizing `identity:get_user(user_id=739d245637684fdc97adf28ed603fb9c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:01.579489 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507408de-1e0e-4b5a-af51-1e444d12cabd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:01.581407 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 553/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:01:01 2022] GET /identity/v3/users/739d245637684fdc97adf28ed603fb9c => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:01.587973 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472b74-4d7b-4a95-bc50-4475575c3a46 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:01.701569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472b74-4d7b-4a95-bc50-4475575c3a46 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:01.737447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd169096-6f02-4e69-8c5e-3d769959c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:01.738088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd169096-6f02-4e69-8c5e-3d769959c5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:01.738396 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd169096-6f02-4e69-8c5e-3d769959c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:01.738673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd169096-6f02-4e69-8c5e-3d769959c5d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:01.746730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:01.811292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd169096-6f02-4e69-8c5e-3d769959c5d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:01.812741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd169096-6f02-4e69-8c5e-3d769959c5d4 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:01.939293 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 575/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:01:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:01:01.945613 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507408de-1e0e-4b5a-af51-1e444d12cabd None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:02.049979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507408de-1e0e-4b5a-af51-1e444d12cabd None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:02.085102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca864f9-d679-4a6f-a2dd-15e5b435ef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:02.085949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca864f9-d679-4a6f-a2dd-15e5b435ef16 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:02.086262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca864f9-d679-4a6f-a2dd-15e5b435ef16 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:02.086531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca864f9-d679-4a6f-a2dd-15e5b435ef16 None admin] PATH_INFO: `/v3/projects/6b9b4da5eeb24eb39a5400029860a693` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:02.087820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca864f9-d679-4a6f-a2dd-15e5b435ef16 None admin] RBAC: Authorizing `identity:get_project(project_id=6b9b4da5eeb24eb39a5400029860a693)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:02.089244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca864f9-d679-4a6f-a2dd-15e5b435ef16 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:02.090933 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 554/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:01:01 2022] GET /identity/v3/projects/6b9b4da5eeb24eb39a5400029860a693 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:02.097660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd169096-6f02-4e69-8c5e-3d769959c5d4 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:02.147903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:02.211119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd169096-6f02-4e69-8c5e-3d769959c5d4 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:02.245155 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215ef4b-3983-4455-aebd-45167b56fa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:02.245791 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215ef4b-3983-4455-aebd-45167b56fa51 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:02.246067 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215ef4b-3983-4455-aebd-45167b56fa51 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:02.246375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215ef4b-3983-4455-aebd-45167b56fa51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:02.247197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215ef4b-3983-4455-aebd-45167b56fa51 None admin] RBAC: Adding query filter params (role.id=70d38d0a968a4e9585c5873d24cfbbb1, scope.project.id=6b9b4da5eeb24eb39a5400029860a693, user.id=739d245637684fdc97adf28ed603fb9c) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:02.247505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215ef4b-3983-4455-aebd-45167b56fa51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:02.248931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215ef4b-3983-4455-aebd-45167b56fa51 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:02.254902 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 576/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:01:02 2022] GET /identity/v3/role_assignments?user.id=739d245637684fdc97adf28ed603fb9c&scope.project.id=6b9b4da5eeb24eb39a5400029860a693&role.id=70d38d0a968a4e9585c5873d24cfbbb1 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:04.271738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb19d6d-d026-45d7-969e-d82fa39ea927 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:04.272343 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb19d6d-d026-45d7-969e-d82fa39ea927 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:04.272343 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb19d6d-d026-45d7-969e-d82fa39ea927 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:04.273702 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 555/1131] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:04.280037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a085-8782-4e71-a6b7-0c12d09ea9fb None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:04.280767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a085-8782-4e71-a6b7-0c12d09ea9fb None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:04.281212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a085-8782-4e71-a6b7-0c12d09ea9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:04.300915 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-0059a085-8782-4e71-a6b7-0c12d09ea9fb None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:04.302122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0059a085-8782-4e71-a6b7-0c12d09ea9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:04.375308 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 577/1132] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:04.410829 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb19d6d-d026-45d7-969e-d82fa39ea927 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:04.457576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:04.518699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb19d6d-d026-45d7-969e-d82fa39ea927 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:04.553292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa3122-c5c7-473a-a220-e9cb9cb48969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:04.554012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa3122-c5c7-473a-a220-e9cb9cb48969 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:04.554322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa3122-c5c7-473a-a220-e9cb9cb48969 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:04.554597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa3122-c5c7-473a-a220-e9cb9cb48969 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:04.555894 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa3122-c5c7-473a-a220-e9cb9cb48969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:04.557316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa3122-c5c7-473a-a220-e9cb9cb48969 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:04.558918 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 556/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:01:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:04.564921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059a085-8782-4e71-a6b7-0c12d09ea9fb None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:04.598370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:04.666575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059a085-8782-4e71-a6b7-0c12d09ea9fb None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:04.703882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0da5c1-ea01-4306-b209-c0fb5c2376d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:04.704537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0da5c1-ea01-4306-b209-c0fb5c2376d1 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:04.704820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0da5c1-ea01-4306-b209-c0fb5c2376d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:04.705392 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0da5c1-ea01-4306-b209-c0fb5c2376d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:04.706118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0da5c1-ea01-4306-b209-c0fb5c2376d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:04.707584 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0da5c1-ea01-4306-b209-c0fb5c2376d1 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:04.739987 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 578/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 10:01:04 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:01:06.744271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8431a-82ea-454e-9eee-d120e326d564 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:06.744880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8431a-82ea-454e-9eee-d120e326d564 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:06.744880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8431a-82ea-454e-9eee-d120e326d564 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:06.746176 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 557/1135] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:06.752681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc6c39-72d3-43f2-8bc2-187de84e9dbc None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:06.753349 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc6c39-72d3-43f2-8bc2-187de84e9dbc None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:06.753442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc6c39-72d3-43f2-8bc2-187de84e9dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:06.780556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-cbfc6c39-72d3-43f2-8bc2-187de84e9dbc None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:06.782331 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-cbfc6c39-72d3-43f2-8bc2-187de84e9dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:06.851772 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 579/1136] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:06.878965 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8431a-82ea-454e-9eee-d120e326d564 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:06.987987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8431a-82ea-454e-9eee-d120e326d564 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:06.995318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:07.030164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7563e9-bb5b-428e-b563-740b9ee48197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:07.031015 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7563e9-bb5b-428e-b563-740b9ee48197 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:07.031301 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7563e9-bb5b-428e-b563-740b9ee48197 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:07.031580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7563e9-bb5b-428e-b563-740b9ee48197 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:07.032842 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7563e9-bb5b-428e-b563-740b9ee48197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:07.034313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7563e9-bb5b-428e-b563-740b9ee48197 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:07.035945 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 558/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:01:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:07.042122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc6c39-72d3-43f2-8bc2-187de84e9dbc None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:07.058090 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:07.158628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc6c39-72d3-43f2-8bc2-187de84e9dbc None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:07.195292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032eedab-d332-4533-bc72-d35cb52e3bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:07.195956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eedab-d332-4533-bc72-d35cb52e3bac None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:07.196240 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eedab-d332-4533-bc72-d35cb52e3bac None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:07.196510 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eedab-d332-4533-bc72-d35cb52e3bac None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:07.197212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032eedab-d332-4533-bc72-d35cb52e3bac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:07.198674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032eedab-d332-4533-bc72-d35cb52e3bac None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:07.222143 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 580/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 10:01:07 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:01:09.239506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465ecd-9be7-4046-9a3b-ae39a1102eca None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:09.240344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465ecd-9be7-4046-9a3b-ae39a1102eca None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:09.240344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465ecd-9be7-4046-9a3b-ae39a1102eca None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:09.242485 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 559/1139] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:09.251201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330368-6062-421e-8e6d-dc4fdfce92e9 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:09.251712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330368-6062-421e-8e6d-dc4fdfce92e9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:09.251811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330368-6062-421e-8e6d-dc4fdfce92e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:09.273783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-88330368-6062-421e-8e6d-dc4fdfce92e9 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:09.274998 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-88330368-6062-421e-8e6d-dc4fdfce92e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:09.339398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:09.345156 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 581/1140] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:09.368953 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2465ecd-9be7-4046-9a3b-ae39a1102eca None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:09.488324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2465ecd-9be7-4046-9a3b-ae39a1102eca None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:09.525894 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abed47-f84b-45c5-bd24-53fbf9f2a5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:09.527182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abed47-f84b-45c5-bd24-53fbf9f2a5de None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:09.527548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abed47-f84b-45c5-bd24-53fbf9f2a5de None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:09.527910 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abed47-f84b-45c5-bd24-53fbf9f2a5de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:09.540254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abed47-f84b-45c5-bd24-53fbf9f2a5de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-73abed47-f84b-45c5-bd24-53fbf9f2a5de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:01:09.547820 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:01:09.549027 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:01:09.549962 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 560/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:01:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:01:09.555017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88330368-6062-421e-8e6d-dc4fdfce92e9 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:09.658786 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88330368-6062-421e-8e6d-dc4fdfce92e9 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:09.693412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83eb7ae-91f2-4317-a272-a74f661a2567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:09.694208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eb7ae-91f2-4317-a272-a74f661a2567 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:09.694519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eb7ae-91f2-4317-a272-a74f661a2567 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:09.694792 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eb7ae-91f2-4317-a272-a74f661a2567 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:09.695536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83eb7ae-91f2-4317-a272-a74f661a2567 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:09.695839 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83eb7ae-91f2-4317-a272-a74f661a2567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:09.697288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83eb7ae-91f2-4317-a272-a74f661a2567 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:09.705571 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 582/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:01:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:09.711847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abed47-f84b-45c5-bd24-53fbf9f2a5de None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:09.750885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:09.834578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abed47-f84b-45c5-bd24-53fbf9f2a5de None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:09.868921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f434d-739e-4d6f-a41d-00cea42eeda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:09.869793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f434d-739e-4d6f-a41d-00cea42eeda2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:09.870071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f434d-739e-4d6f-a41d-00cea42eeda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:09.870376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f434d-739e-4d6f-a41d-00cea42eeda2 None admin] PATH_INFO: `/v3/users/c62bacceeff64055997f8a5da37649d0` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:09.887973 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f434d-739e-4d6f-a41d-00cea42eeda2 None admin] RBAC: Authorizing `identity:get_user(user_id=c62bacceeff64055997f8a5da37649d0)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:09.890208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f434d-739e-4d6f-a41d-00cea42eeda2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:09.892091 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 561/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:01:09 2022] GET /identity/v3/users/c62bacceeff64055997f8a5da37649d0 => generated 286 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:09.898156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83eb7ae-91f2-4317-a272-a74f661a2567 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:09.966807 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:10.008367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83eb7ae-91f2-4317-a272-a74f661a2567 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:10.043084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568a586-b771-44d1-b6d3-058da166c56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:10.043724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568a586-b771-44d1-b6d3-058da166c56b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:10.044002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568a586-b771-44d1-b6d3-058da166c56b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:10.044282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568a586-b771-44d1-b6d3-058da166c56b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:10.118802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568a586-b771-44d1-b6d3-058da166c56b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:10.120230 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568a586-b771-44d1-b6d3-058da166c56b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:10.179173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:10.219389 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 583/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:01:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:01:10.229225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f434d-739e-4d6f-a41d-00cea42eeda2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:10.341436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:10.351336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f434d-739e-4d6f-a41d-00cea42eeda2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:10.388274 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b206f-c348-4010-b67c-d4ad284ef66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:10.389068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b206f-c348-4010-b67c-d4ad284ef66c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:10.389362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b206f-c348-4010-b67c-d4ad284ef66c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:10.389629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b206f-c348-4010-b67c-d4ad284ef66c None admin] PATH_INFO: `/v3/projects/93a0a2db9a8c4aababf27003a6e1c37c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:10.390909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b206f-c348-4010-b67c-d4ad284ef66c None admin] RBAC: Authorizing `identity:get_project(project_id=93a0a2db9a8c4aababf27003a6e1c37c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:10.392337 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b206f-c348-4010-b67c-d4ad284ef66c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:10.394012 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 562/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:01:10 2022] GET /identity/v3/projects/93a0a2db9a8c4aababf27003a6e1c37c => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:10.400312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568a586-b771-44d1-b6d3-058da166c56b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:10.504770 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568a586-b771-44d1-b6d3-058da166c56b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:10.546264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe50a35-6966-4b2f-8175-2fa37e9b2cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:10.546921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe50a35-6966-4b2f-8175-2fa37e9b2cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:10.547209 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe50a35-6966-4b2f-8175-2fa37e9b2cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:10.547485 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe50a35-6966-4b2f-8175-2fa37e9b2cf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:10.548322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe50a35-6966-4b2f-8175-2fa37e9b2cf6 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=93a0a2db9a8c4aababf27003a6e1c37c, user.id=c62bacceeff64055997f8a5da37649d0) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:10.548634 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe50a35-6966-4b2f-8175-2fa37e9b2cf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:10.550110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe50a35-6966-4b2f-8175-2fa37e9b2cf6 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:10.555555 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 584/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:01:10 2022] GET /identity/v3/role_assignments?user.id=c62bacceeff64055997f8a5da37649d0&scope.project.id=93a0a2db9a8c4aababf27003a6e1c37c&role.id=c184a177010144429f495f3290d28221 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:12.532370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff3aa4-b639-46b6-9329-ff0f6f5d8d66 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:12.533266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff3aa4-b639-46b6-9329-ff0f6f5d8d66 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:12.533266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff3aa4-b639-46b6-9329-ff0f6f5d8d66 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:12.535491 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 563/1147] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:12.541718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e8c3-699c-4159-a393-3f26146551d3 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:12.542156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e8c3-699c-4159-a393-3f26146551d3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:12.542348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e8c3-699c-4159-a393-3f26146551d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:12.567511 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-f139e8c3-699c-4159-a393-3f26146551d3 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:12.568693 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f139e8c3-699c-4159-a393-3f26146551d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:12.590290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:12.640185 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 585/1148] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:12.668746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff3aa4-b639-46b6-9329-ff0f6f5d8d66 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:12.777672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff3aa4-b639-46b6-9329-ff0f6f5d8d66 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:12.812149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf07c13-a914-411c-ad3d-bd7c270960d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:12.812915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf07c13-a914-411c-ad3d-bd7c270960d3 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:12.813192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf07c13-a914-411c-ad3d-bd7c270960d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:12.813469 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf07c13-a914-411c-ad3d-bd7c270960d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:12.823774 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf07c13-a914-411c-ad3d-bd7c270960d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-1cf07c13-a914-411c-ad3d-bd7c270960d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:01:12.831692 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:01:12.832748 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:01:12.833958 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 564/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:01:12.842580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139e8c3-699c-4159-a393-3f26146551d3 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:12.954281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139e8c3-699c-4159-a393-3f26146551d3 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:12.989528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd9e4e-67e5-4f3a-99b2-ce5a974e162e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:12.990177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd9e4e-67e5-4f3a-99b2-ce5a974e162e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:12.990491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd9e4e-67e5-4f3a-99b2-ce5a974e162e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:12.990764 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd9e4e-67e5-4f3a-99b2-ce5a974e162e None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:12.991512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd9e4e-67e5-4f3a-99b2-ce5a974e162e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:12.991816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd9e4e-67e5-4f3a-99b2-ce5a974e162e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:12.993200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd9e4e-67e5-4f3a-99b2-ce5a974e162e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:13.001553 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 586/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:13.007591 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf07c13-a914-411c-ad3d-bd7c270960d3 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:13.033541 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:13.119045 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf07c13-a914-411c-ad3d-bd7c270960d3 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:13.159846 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db2b74-1fe9-4887-b583-ed47bf592ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:13.160714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db2b74-1fe9-4887-b583-ed47bf592ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:13.160996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db2b74-1fe9-4887-b583-ed47bf592ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:13.161271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db2b74-1fe9-4887-b583-ed47bf592ce1 None admin] PATH_INFO: `/v3/users/c62bacceeff64055997f8a5da37649d0` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:13.162626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db2b74-1fe9-4887-b583-ed47bf592ce1 None admin] RBAC: Authorizing `identity:get_user(user_id=c62bacceeff64055997f8a5da37649d0)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:13.164060 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db2b74-1fe9-4887-b583-ed47bf592ce1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:13.165695 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 565/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:01:13 2022] GET /identity/v3/users/c62bacceeff64055997f8a5da37649d0 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:13.174364 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd9e4e-67e5-4f3a-99b2-ce5a974e162e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:13.195417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:13.280230 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd9e4e-67e5-4f3a-99b2-ce5a974e162e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:13.315658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba37b3-6be6-4fa1-bb71-58f7cf0741b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:13.316523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba37b3-6be6-4fa1-bb71-58f7cf0741b5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:13.316815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba37b3-6be6-4fa1-bb71-58f7cf0741b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:13.317115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba37b3-6be6-4fa1-bb71-58f7cf0741b5 None admin] PATH_INFO: `/v3/projects/93a0a2db9a8c4aababf27003a6e1c37c` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:13.318442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba37b3-6be6-4fa1-bb71-58f7cf0741b5 None admin] RBAC: Authorizing `identity:get_project(project_id=93a0a2db9a8c4aababf27003a6e1c37c)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:13.319892 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba37b3-6be6-4fa1-bb71-58f7cf0741b5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:13.321568 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 587/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:01:13 2022] GET /identity/v3/projects/93a0a2db9a8c4aababf27003a6e1c37c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:13.331177 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db2b74-1fe9-4887-b583-ed47bf592ce1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:13.429771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:13.445143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db2b74-1fe9-4887-b583-ed47bf592ce1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:13.480451 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757e642-18a2-450f-9c70-6467089b8062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:13.481333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757e642-18a2-450f-9c70-6467089b8062 None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:13.481628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757e642-18a2-450f-9c70-6467089b8062 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:13.481891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757e642-18a2-450f-9c70-6467089b8062 None admin] PATH_INFO: `/v3/projects/93a0a2db9a8c4aababf27003a6e1c37c/users/c62bacceeff64055997f8a5da37649d0/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:13.484119 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757e642-18a2-450f-9c70-6467089b8062 None admin] RBAC: Authorizing `identity:create_grant(project_id=93a0a2db9a8c4aababf27003a6e1c37c, user_id=c62bacceeff64055997f8a5da37649d0, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:13.485545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757e642-18a2-450f-9c70-6467089b8062 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:13.501266 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 566/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 10:01:13 2022] PUT /identity/v3/projects/93a0a2db9a8c4aababf27003a6e1c37c/users/c62bacceeff64055997f8a5da37649d0/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:01:15.468169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32662260-9c16-4000-9790-44210a07bbcc None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:15.469140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32662260-9c16-4000-9790-44210a07bbcc None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:15.469265 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32662260-9c16-4000-9790-44210a07bbcc None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:15.471586 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 588/1154] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:15.476931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42661b-1388-474a-9d00-10b5ab8b758f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:15.477368 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42661b-1388-474a-9d00-10b5ab8b758f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:15.477475 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42661b-1388-474a-9d00-10b5ab8b758f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:15.503735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4c42661b-1388-474a-9d00-10b5ab8b758f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:15.505111 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4c42661b-1388-474a-9d00-10b5ab8b758f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:15.579877 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 567/1155] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:15.603347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32662260-9c16-4000-9790-44210a07bbcc None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:15.694587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:15.713663 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32662260-9c16-4000-9790-44210a07bbcc None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:15.749551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263401a-25c9-4382-96c9-afe0fb686624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:15.750483 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263401a-25c9-4382-96c9-afe0fb686624 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:15.750765 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263401a-25c9-4382-96c9-afe0fb686624 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:15.751034 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263401a-25c9-4382-96c9-afe0fb686624 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:15.763823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263401a-25c9-4382-96c9-afe0fb686624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-2263401a-25c9-4382-96c9-afe0fb686624 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:01:15.771417 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:01:15.772497 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:15.773506 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:01:15.780462 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 589/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:01:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:01:15.780604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42661b-1388-474a-9d00-10b5ab8b758f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:15.858937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:15.893642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42661b-1388-474a-9d00-10b5ab8b758f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:15.929898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb995568-256e-4144-a907-122d466cfee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:15.930593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb995568-256e-4144-a907-122d466cfee4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:15.930869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb995568-256e-4144-a907-122d466cfee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:15.931143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb995568-256e-4144-a907-122d466cfee4 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:15.931891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb995568-256e-4144-a907-122d466cfee4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:15.932197 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb995568-256e-4144-a907-122d466cfee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:15.933568 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb995568-256e-4144-a907-122d466cfee4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:15.941781 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 568/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:01:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:15.947723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263401a-25c9-4382-96c9-afe0fb686624 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:16.053833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263401a-25c9-4382-96c9-afe0fb686624 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:16.087328 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590e939-e267-4f8d-8173-248cbbd3b7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:16.088197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590e939-e267-4f8d-8173-248cbbd3b7ab None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:16.088475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590e939-e267-4f8d-8173-248cbbd3b7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:16.088748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590e939-e267-4f8d-8173-248cbbd3b7ab None admin] PATH_INFO: `/v3/users/c62bacceeff64055997f8a5da37649d0` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:16.089948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590e939-e267-4f8d-8173-248cbbd3b7ab None admin] RBAC: Authorizing `identity:get_user(user_id=c62bacceeff64055997f8a5da37649d0)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:16.091412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590e939-e267-4f8d-8173-248cbbd3b7ab None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:16.093035 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 590/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:01:15 2022] GET /identity/v3/users/c62bacceeff64055997f8a5da37649d0 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:16.099065 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb995568-256e-4144-a907-122d466cfee4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:16.213849 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb995568-256e-4144-a907-122d466cfee4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:16.252942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7eca59-d318-486c-8754-0c3dafda9e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:16.253588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eca59-d318-486c-8754-0c3dafda9e77 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:16.253874 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eca59-d318-486c-8754-0c3dafda9e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:16.254158 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eca59-d318-486c-8754-0c3dafda9e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:16.264758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:16.330031 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7eca59-d318-486c-8754-0c3dafda9e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:16.331436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7eca59-d318-486c-8754-0c3dafda9e77 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:16.433154 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 569/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:01:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:01:16.439513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590e939-e267-4f8d-8173-248cbbd3b7ab None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:16.458056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:16.548107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590e939-e267-4f8d-8173-248cbbd3b7ab None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:16.583394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f757e36-d7c6-43b2-9b47-b1938768f1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:16.584182 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f757e36-d7c6-43b2-9b47-b1938768f1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:16.584461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f757e36-d7c6-43b2-9b47-b1938768f1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:16.584738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f757e36-d7c6-43b2-9b47-b1938768f1f0 None admin] PATH_INFO: `/v3/projects/93a0a2db9a8c4aababf27003a6e1c37c` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:16.586032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f757e36-d7c6-43b2-9b47-b1938768f1f0 None admin] RBAC: Authorizing `identity:get_project(project_id=93a0a2db9a8c4aababf27003a6e1c37c)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:16.587525 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f757e36-d7c6-43b2-9b47-b1938768f1f0 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:16.589206 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 591/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:01:16 2022] GET /identity/v3/projects/93a0a2db9a8c4aababf27003a6e1c37c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:16.598482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7eca59-d318-486c-8754-0c3dafda9e77 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:16.643177 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:16.707808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7eca59-d318-486c-8754-0c3dafda9e77 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:16.743513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc66cc-ead9-4641-bb61-d1fe4c9c0f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:16.744177 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc66cc-ead9-4641-bb61-d1fe4c9c0f94 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:16.744463 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc66cc-ead9-4641-bb61-d1fe4c9c0f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:16.744735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc66cc-ead9-4641-bb61-d1fe4c9c0f94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:16.745580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc66cc-ead9-4641-bb61-d1fe4c9c0f94 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=93a0a2db9a8c4aababf27003a6e1c37c, user.id=c62bacceeff64055997f8a5da37649d0) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:16.745890 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc66cc-ead9-4641-bb61-d1fe4c9c0f94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:16.747386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc66cc-ead9-4641-bb61-d1fe4c9c0f94 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:16.753385 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 570/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:01:16 2022] GET /identity/v3/role_assignments?user.id=c62bacceeff64055997f8a5da37649d0&scope.project.id=93a0a2db9a8c4aababf27003a6e1c37c&role.id=c184a177010144429f495f3290d28221 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:18.754994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f46da-f748-46f7-87f1-e10e00d19b41 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:18.755553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f46da-f748-46f7-87f1-e10e00d19b41 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:18.755553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f46da-f748-46f7-87f1-e10e00d19b41 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:18.756975 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 592/1162] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:18.763628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a476f-23dc-466a-8428-a1c71491ee70 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:18.764318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a476f-23dc-466a-8428-a1c71491ee70 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:18.764492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a476f-23dc-466a-8428-a1c71491ee70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:18.787941 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b96a476f-23dc-466a-8428-a1c71491ee70 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:18.789146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b96a476f-23dc-466a-8428-a1c71491ee70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:18.867395 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 571/1163] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:18.890303 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f46da-f748-46f7-87f1-e10e00d19b41 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:18.989216 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:19.009851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f46da-f748-46f7-87f1-e10e00d19b41 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:19.043677 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce19ac-b0fb-4d61-81c6-c032c2c25630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:19.044325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce19ac-b0fb-4d61-81c6-c032c2c25630 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:19.044603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce19ac-b0fb-4d61-81c6-c032c2c25630 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:19.044875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce19ac-b0fb-4d61-81c6-c032c2c25630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:19.115016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce19ac-b0fb-4d61-81c6-c032c2c25630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:19.116457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce19ac-b0fb-4d61-81c6-c032c2c25630 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:19.206983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:19.227713 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 593/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:01:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:01:19.234249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a476f-23dc-466a-8428-a1c71491ee70 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:19.269251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:19.335351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a476f-23dc-466a-8428-a1c71491ee70 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:19.371183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed87775-76b4-4630-8d3e-9ce2d090f6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:19.371977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed87775-76b4-4630-8d3e-9ce2d090f6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:19.372259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed87775-76b4-4630-8d3e-9ce2d090f6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:19.372543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed87775-76b4-4630-8d3e-9ce2d090f6e4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:19.380085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed87775-76b4-4630-8d3e-9ce2d090f6e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:19.381537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed87775-76b4-4630-8d3e-9ce2d090f6e4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9ed87775-76b4-4630-8d3e-9ce2d090f6e4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:01:19.396609 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:01:19.397788 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:01:19.398819 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 572/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jul 5 10:01:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:01:19.404336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce19ac-b0fb-4d61-81c6-c032c2c25630 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:19.514146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce19ac-b0fb-4d61-81c6-c032c2c25630 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:19.550103 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd3438-fadd-4da8-a6f7-3012a2159e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:19.550717 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd3438-fadd-4da8-a6f7-3012a2159e13 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:19.550998 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd3438-fadd-4da8-a6f7-3012a2159e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:19.551275 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd3438-fadd-4da8-a6f7-3012a2159e13 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:19.551985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd3438-fadd-4da8-a6f7-3012a2159e13 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:19.552289 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd3438-fadd-4da8-a6f7-3012a2159e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:19.553664 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd3438-fadd-4da8-a6f7-3012a2159e13 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:19.560749 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 594/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jul 5 10:01:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:21.553806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b550fb-b4cf-4b16-affa-261c5dcf0b99 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:21.554484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b550fb-b4cf-4b16-affa-261c5dcf0b99 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:21.554484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b550fb-b4cf-4b16-affa-261c5dcf0b99 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:21.555786 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 573/1167] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:21.562064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df91f-bd07-4dad-84f1-e0638b73ebbf None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:21.562543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df91f-bd07-4dad-84f1-e0638b73ebbf None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:21.562637 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df91f-bd07-4dad-84f1-e0638b73ebbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:21.591123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-817df91f-bd07-4dad-84f1-e0638b73ebbf None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:21.593080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-817df91f-bd07-4dad-84f1-e0638b73ebbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:21.642268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:21.669998 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 595/1168] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:21.693592 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b550fb-b4cf-4b16-affa-261c5dcf0b99 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:21.806173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:21.816047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b550fb-b4cf-4b16-affa-261c5dcf0b99 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:21.851658 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b135c-b49b-426f-a409-8d0700b58c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:21.852329 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b135c-b49b-426f-a409-8d0700b58c82 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:21.852611 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b135c-b49b-426f-a409-8d0700b58c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:21.852887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b135c-b49b-426f-a409-8d0700b58c82 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:21.853652 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b135c-b49b-426f-a409-8d0700b58c82 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:21.855051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b135c-b49b-426f-a409-8d0700b58c82 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:21.873615 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 574/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Tue Jul 5 10:01:21 2022] POST /identity/v3/domains => generated 267 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:01:23.895607 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942cec4-0280-4f1a-bd47-ca6067f4ea70 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:23.896201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942cec4-0280-4f1a-bd47-ca6067f4ea70 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:23.896201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942cec4-0280-4f1a-bd47-ca6067f4ea70 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:23.897613 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 596/1170] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:23.904056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2523d5-329a-49ab-8148-4051bd4906fa None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:23.904813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2523d5-329a-49ab-8148-4051bd4906fa None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:23.905026 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2523d5-329a-49ab-8148-4051bd4906fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:23.924618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3d2523d5-329a-49ab-8148-4051bd4906fa None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:23.925830 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3d2523d5-329a-49ab-8148-4051bd4906fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:23.996797 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 575/1171] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:24.020110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942cec4-0280-4f1a-bd47-ca6067f4ea70 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:24.128864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942cec4-0280-4f1a-bd47-ca6067f4ea70 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:24.163329 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf766bed-84ca-4dbb-b15f-b09e105b010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:24.164188 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766bed-84ca-4dbb-b15f-b09e105b010b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:24.164469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766bed-84ca-4dbb-b15f-b09e105b010b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:24.164746 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766bed-84ca-4dbb-b15f-b09e105b010b None admin] PATH_INFO: `/v3/domains/ac8a806b43e945ca848fd8648e8e8a43` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:24.175715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf766bed-84ca-4dbb-b15f-b09e105b010b None admin] RBAC: Authorizing `identity:get_domain(domain_id=ac8a806b43e945ca848fd8648e8e8a43)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:24.177163 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf766bed-84ca-4dbb-b15f-b09e105b010b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:24.178829 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 597/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Tue Jul 5 10:01:24 2022] GET /identity/v3/domains/ac8a806b43e945ca848fd8648e8e8a43 => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:24.184970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2523d5-329a-49ab-8148-4051bd4906fa None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:24.271447 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:24.294792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2523d5-329a-49ab-8148-4051bd4906fa None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:24.331281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9de1c-b48f-4b7b-bdcf-b623620c3992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:24.331956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9de1c-b48f-4b7b-bdcf-b623620c3992 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:24.332235 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9de1c-b48f-4b7b-bdcf-b623620c3992 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:24.332508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9de1c-b48f-4b7b-bdcf-b623620c3992 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:24.333221 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9de1c-b48f-4b7b-bdcf-b623620c3992 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:24.334707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9de1c-b48f-4b7b-bdcf-b623620c3992 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:24.364585 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 576/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 10:01:24 2022] POST /identity/v3/projects => generated 361 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:01:26.394766 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61016891-b3d9-470f-992e-4ec7a5e8d61d None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:26.395368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61016891-b3d9-470f-992e-4ec7a5e8d61d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:26.395368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61016891-b3d9-470f-992e-4ec7a5e8d61d None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:26.396757 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 598/1174] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:26.403287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b84a4-6368-4611-8f70-3e75ff57442a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:26.404066 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b84a4-6368-4611-8f70-3e75ff57442a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:26.404210 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b84a4-6368-4611-8f70-3e75ff57442a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:26.427846 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-aa7b84a4-6368-4611-8f70-3e75ff57442a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:26.429047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-aa7b84a4-6368-4611-8f70-3e75ff57442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:26.498139 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 577/1175] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:26.522920 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61016891-b3d9-470f-992e-4ec7a5e8d61d None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:26.566573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:26.639248 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61016891-b3d9-470f-992e-4ec7a5e8d61d None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:26.681799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7ba00-8a4c-471f-8a55-fd8e710f449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:26.682648 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7ba00-8a4c-471f-8a55-fd8e710f449c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:26.682934 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7ba00-8a4c-471f-8a55-fd8e710f449c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:26.683207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7ba00-8a4c-471f-8a55-fd8e710f449c None admin] PATH_INFO: `/v3/domains/ac8a806b43e945ca848fd8648e8e8a43` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:26.684446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7ba00-8a4c-471f-8a55-fd8e710f449c None admin] RBAC: Authorizing `identity:get_domain(domain_id=ac8a806b43e945ca848fd8648e8e8a43)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:26.685902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7ba00-8a4c-471f-8a55-fd8e710f449c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:26.687658 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 599/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Tue Jul 5 10:01:26 2022] GET /identity/v3/domains/ac8a806b43e945ca848fd8648e8e8a43 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:26.694004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b84a4-6368-4611-8f70-3e75ff57442a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:26.773383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:26.809930 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b84a4-6368-4611-8f70-3e75ff57442a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:26.851666 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd73c23-7ef3-4048-b197-2af3fb07bb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:26.852341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd73c23-7ef3-4048-b197-2af3fb07bb23 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:26.852625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd73c23-7ef3-4048-b197-2af3fb07bb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:26.852900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd73c23-7ef3-4048-b197-2af3fb07bb23 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:26.853609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd73c23-7ef3-4048-b197-2af3fb07bb23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:26.855074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd73c23-7ef3-4048-b197-2af3fb07bb23 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:26.879643 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 578/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 10:01:26 2022] POST /identity/v3/users => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:01:28.883758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cad11-7913-40c5-8528-09ac7359c1f8 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:28.884587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cad11-7913-40c5-8528-09ac7359c1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:28.884587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cad11-7913-40c5-8528-09ac7359c1f8 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:28.887102 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 600/1178] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:28.893714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e377e-d484-4d36-b344-78866a2cc837 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:28.894152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e377e-d484-4d36-b344-78866a2cc837 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:28.894269 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e377e-d484-4d36-b344-78866a2cc837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:28.912719 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-700e377e-d484-4d36-b344-78866a2cc837 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:28.913892 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-700e377e-d484-4d36-b344-78866a2cc837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:28.984810 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 579/1179] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 10:01:29.008037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cad11-7913-40c5-8528-09ac7359c1f8 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:29.125420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cad11-7913-40c5-8528-09ac7359c1f8 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:29.127868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:29.166641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32028058-847c-43bb-85b1-9665ad3bae77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:29.167431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32028058-847c-43bb-85b1-9665ad3bae77 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:29.167718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32028058-847c-43bb-85b1-9665ad3bae77 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:29.167992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32028058-847c-43bb-85b1-9665ad3bae77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:29.178582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32028058-847c-43bb-85b1-9665ad3bae77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-32028058-847c-43bb-85b1-9665ad3bae77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:01:29.186128 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:01:29.187256 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:01:29.188350 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 601/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:01:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:01:29.193485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e377e-d484-4d36-b344-78866a2cc837 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:29.251244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:29.305973 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e377e-d484-4d36-b344-78866a2cc837 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:29.343705 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6943ee-f8df-48d9-86b0-06036b5afac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:29.344366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6943ee-f8df-48d9-86b0-06036b5afac8 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:29.344651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6943ee-f8df-48d9-86b0-06036b5afac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:29.344927 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6943ee-f8df-48d9-86b0-06036b5afac8 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:29.345684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6943ee-f8df-48d9-86b0-06036b5afac8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:29.345995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6943ee-f8df-48d9-86b0-06036b5afac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:29.347406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6943ee-f8df-48d9-86b0-06036b5afac8 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:29.355606 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 580/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:01:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:01:29.363356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32028058-847c-43bb-85b1-9665ad3bae77 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:29.476992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32028058-847c-43bb-85b1-9665ad3bae77 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:29.511179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360752c3-2db7-48d0-9670-5dd7a6aafa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:29.512040 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360752c3-2db7-48d0-9670-5dd7a6aafa24 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:29.512321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360752c3-2db7-48d0-9670-5dd7a6aafa24 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:29.512593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360752c3-2db7-48d0-9670-5dd7a6aafa24 None admin] PATH_INFO: `/v3/users/55dfc051d67643ea9db15c8fe807ba02` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:29.527173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360752c3-2db7-48d0-9670-5dd7a6aafa24 None admin] RBAC: Authorizing `identity:get_user(user_id=55dfc051d67643ea9db15c8fe807ba02)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:29.528649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360752c3-2db7-48d0-9670-5dd7a6aafa24 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:29.530425 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 602/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:01:29 2022] GET /identity/v3/users/55dfc051d67643ea9db15c8fe807ba02 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:29.538960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6943ee-f8df-48d9-86b0-06036b5afac8 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:29.641041 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6943ee-f8df-48d9-86b0-06036b5afac8 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:29.654723 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:29.676307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2c7ca-79a8-4f89-ae08-7554e59b5c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:29.676958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2c7ca-79a8-4f89-ae08-7554e59b5c5b None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:29.677244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2c7ca-79a8-4f89-ae08-7554e59b5c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:29.677530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2c7ca-79a8-4f89-ae08-7554e59b5c5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:29.754435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2c7ca-79a8-4f89-ae08-7554e59b5c5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:29.755887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2c7ca-79a8-4f89-ae08-7554e59b5c5b None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:29.860324 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 581/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:01:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 10:01:29.866570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360752c3-2db7-48d0-9670-5dd7a6aafa24 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:29.895024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:29.983437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360752c3-2db7-48d0-9670-5dd7a6aafa24 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:30.019656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eaf0aa-a713-4894-a429-528092734a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:30.020452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaf0aa-a713-4894-a429-528092734a5b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:30.020760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaf0aa-a713-4894-a429-528092734a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:30.021056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaf0aa-a713-4894-a429-528092734a5b None admin] PATH_INFO: `/v3/projects/21f64426677041df90d9f8960f310de1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:30.022358 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaf0aa-a713-4894-a429-528092734a5b None admin] RBAC: Authorizing `identity:get_project(project_id=21f64426677041df90d9f8960f310de1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:30.023796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaf0aa-a713-4894-a429-528092734a5b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:30.025488 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 603/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:01:29 2022] GET /identity/v3/projects/21f64426677041df90d9f8960f310de1 => generated 361 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:30.031956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2c7ca-79a8-4f89-ae08-7554e59b5c5b None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:30.049554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:30.140075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2c7ca-79a8-4f89-ae08-7554e59b5c5b None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:30.176514 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b073b-d472-4885-b301-be529a1e8a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:30.177164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b073b-d472-4885-b301-be529a1e8a22 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:30.177450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b073b-d472-4885-b301-be529a1e8a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:30.177732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b073b-d472-4885-b301-be529a1e8a22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:30.178596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b073b-d472-4885-b301-be529a1e8a22 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=21f64426677041df90d9f8960f310de1, user.id=55dfc051d67643ea9db15c8fe807ba02) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:30.178910 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b073b-d472-4885-b301-be529a1e8a22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:30.180346 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b073b-d472-4885-b301-be529a1e8a22 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:30.186254 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 582/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:01:30 2022] GET /identity/v3/role_assignments?user.id=55dfc051d67643ea9db15c8fe807ba02&scope.project.id=21f64426677041df90d9f8960f310de1&role.id=c184a177010144429f495f3290d28221 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:32.168639 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd25a6-8a98-45de-9139-3c235f61c85a None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:32.169559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd25a6-8a98-45de-9139-3c235f61c85a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:32.169559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd25a6-8a98-45de-9139-3c235f61c85a None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:32.171701 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 604/1186] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:32.177925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c087-4ee7-4da7-9082-daf9d3261f2e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:32.178416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c087-4ee7-4da7-9082-daf9d3261f2e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:32.178521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c087-4ee7-4da7-9082-daf9d3261f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:32.199044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1e41c087-4ee7-4da7-9082-daf9d3261f2e None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:32.200225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1e41c087-4ee7-4da7-9082-daf9d3261f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:32.269874 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 583/1187] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:32.292919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd25a6-8a98-45de-9139-3c235f61c85a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:32.391117 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:32.402826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd25a6-8a98-45de-9139-3c235f61c85a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:32.441520 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2d55f-fdc9-4438-8e69-c8ca8a4de3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:32.442639 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2d55f-fdc9-4438-8e69-c8ca8a4de3bb None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:32.442982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2d55f-fdc9-4438-8e69-c8ca8a4de3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:32.443394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2d55f-fdc9-4438-8e69-c8ca8a4de3bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:32.455159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2d55f-fdc9-4438-8e69-c8ca8a4de3bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-99c2d55f-fdc9-4438-8e69-c8ca8a4de3bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:01:32.462738 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:01:32.463888 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:01:32.465124 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 605/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:01:32.472067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41c087-4ee7-4da7-9082-daf9d3261f2e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:32.486119 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:32.580081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41c087-4ee7-4da7-9082-daf9d3261f2e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:32.618819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb93a3-5ca9-4c29-9402-3cc67e4eec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:32.619513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb93a3-5ca9-4c29-9402-3cc67e4eec9d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:32.619799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb93a3-5ca9-4c29-9402-3cc67e4eec9d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:32.620074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb93a3-5ca9-4c29-9402-3cc67e4eec9d None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:32.620847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb93a3-5ca9-4c29-9402-3cc67e4eec9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:32.621150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb93a3-5ca9-4c29-9402-3cc67e4eec9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:32.622561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb93a3-5ca9-4c29-9402-3cc67e4eec9d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:32.630911 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 584/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:32.637361 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2d55f-fdc9-4438-8e69-c8ca8a4de3bb None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:32.743729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2d55f-fdc9-4438-8e69-c8ca8a4de3bb None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:32.779284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8ddf2-e2c0-4a76-a6e2-972a0228177b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:32.780192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8ddf2-e2c0-4a76-a6e2-972a0228177b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:32.780462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8ddf2-e2c0-4a76-a6e2-972a0228177b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:32.780733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8ddf2-e2c0-4a76-a6e2-972a0228177b None admin] PATH_INFO: `/v3/users/55dfc051d67643ea9db15c8fe807ba02` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:32.781963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8ddf2-e2c0-4a76-a6e2-972a0228177b None admin] RBAC: Authorizing `identity:get_user(user_id=55dfc051d67643ea9db15c8fe807ba02)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:32.783462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8ddf2-e2c0-4a76-a6e2-972a0228177b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:32.785087 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 606/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:01:32 2022] GET /identity/v3/users/55dfc051d67643ea9db15c8fe807ba02 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:32.794503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb93a3-5ca9-4c29-9402-3cc67e4eec9d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:32.871782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:32.903559 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb93a3-5ca9-4c29-9402-3cc67e4eec9d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:32.937804 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6176f6-98ff-4c14-807a-257a22119b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:32.938689 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6176f6-98ff-4c14-807a-257a22119b55 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:32.938973 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6176f6-98ff-4c14-807a-257a22119b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:32.939244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6176f6-98ff-4c14-807a-257a22119b55 None admin] PATH_INFO: `/v3/projects/21f64426677041df90d9f8960f310de1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:32.940556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6176f6-98ff-4c14-807a-257a22119b55 None admin] RBAC: Authorizing `identity:get_project(project_id=21f64426677041df90d9f8960f310de1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:32.942002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6176f6-98ff-4c14-807a-257a22119b55 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:32.943774 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 585/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:01:32 2022] GET /identity/v3/projects/21f64426677041df90d9f8960f310de1 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:32.952759 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8ddf2-e2c0-4a76-a6e2-972a0228177b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:32.979209 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:33.061322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8ddf2-e2c0-4a76-a6e2-972a0228177b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:33.103303 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05857ec-ed51-46f0-8325-e6f3ba5dc57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:33.104197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05857ec-ed51-46f0-8325-e6f3ba5dc57b None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:33.104485 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05857ec-ed51-46f0-8325-e6f3ba5dc57b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:33.104760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05857ec-ed51-46f0-8325-e6f3ba5dc57b None admin] PATH_INFO: `/v3/projects/21f64426677041df90d9f8960f310de1/users/55dfc051d67643ea9db15c8fe807ba02/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:33.106903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05857ec-ed51-46f0-8325-e6f3ba5dc57b None admin] RBAC: Authorizing `identity:create_grant(project_id=21f64426677041df90d9f8960f310de1, user_id=55dfc051d67643ea9db15c8fe807ba02, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:33.108442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05857ec-ed51-46f0-8325-e6f3ba5dc57b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:33.124103 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 607/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 10:01:32 2022] PUT /identity/v3/projects/21f64426677041df90d9f8960f310de1/users/55dfc051d67643ea9db15c8fe807ba02/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:01:35.114214 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d8bdf-d9e4-4b1f-9936-66f20291c4ab None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:35.114805 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d8bdf-d9e4-4b1f-9936-66f20291c4ab None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:35.114805 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d8bdf-d9e4-4b1f-9936-66f20291c4ab None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:35.116084 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 586/1193] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:01:35.121463 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372fbc3-e058-44e0-a055-d680d2da9547 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:35.121897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372fbc3-e058-44e0-a055-d680d2da9547 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:35.121996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372fbc3-e058-44e0-a055-d680d2da9547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:35.140567 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-c372fbc3-e058-44e0-a055-d680d2da9547 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:01:35.141737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c372fbc3-e058-44e0-a055-d680d2da9547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:01:35.213765 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 608/1194] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:01:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:01:35.238742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d8bdf-d9e4-4b1f-9936-66f20291c4ab None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:35.354730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d8bdf-d9e4-4b1f-9936-66f20291c4ab None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:35.390589 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9387cc1-0a87-4b78-bd4f-ce2dca929f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:35.391332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387cc1-0a87-4b78-bd4f-ce2dca929f06 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:35.391615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387cc1-0a87-4b78-bd4f-ce2dca929f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:35.391890 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9387cc1-0a87-4b78-bd4f-ce2dca929f06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:35.399051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:35.402701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9387cc1-0a87-4b78-bd4f-ce2dca929f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d9387cc1-0a87-4b78-bd4f-ce2dca929f06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:01:35.411070 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:01:35.412211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:01:35.413101 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 587/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:01:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:01:35.419449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372fbc3-e058-44e0-a055-d680d2da9547 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:35.430527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:35.522517 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372fbc3-e058-44e0-a055-d680d2da9547 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:35.555886 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0301bd-bd9b-473b-82e8-12adff8bb1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:35.556543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0301bd-bd9b-473b-82e8-12adff8bb1ae None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:35.556827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0301bd-bd9b-473b-82e8-12adff8bb1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:35.557100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0301bd-bd9b-473b-82e8-12adff8bb1ae None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:35.557843 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0301bd-bd9b-473b-82e8-12adff8bb1ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:35.558152 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0301bd-bd9b-473b-82e8-12adff8bb1ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:35.559580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0301bd-bd9b-473b-82e8-12adff8bb1ae None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:35.567779 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 609/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:35.574524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9387cc1-0a87-4b78-bd4f-ce2dca929f06 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:35.684213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9387cc1-0a87-4b78-bd4f-ce2dca929f06 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:35.718787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee32c3-5ddb-4bff-af3f-087d53f60030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:35.719666 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee32c3-5ddb-4bff-af3f-087d53f60030 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:35.719941 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee32c3-5ddb-4bff-af3f-087d53f60030 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:35.720215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee32c3-5ddb-4bff-af3f-087d53f60030 None admin] PATH_INFO: `/v3/users/55dfc051d67643ea9db15c8fe807ba02` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:35.721445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee32c3-5ddb-4bff-af3f-087d53f60030 None admin] RBAC: Authorizing `identity:get_user(user_id=55dfc051d67643ea9db15c8fe807ba02)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:35.722920 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee32c3-5ddb-4bff-af3f-087d53f60030 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:35.724524 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 588/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jul 5 10:01:35 2022] GET /identity/v3/users/55dfc051d67643ea9db15c8fe807ba02 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:35.734258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0301bd-bd9b-473b-82e8-12adff8bb1ae None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:35.809132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:35.840385 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0301bd-bd9b-473b-82e8-12adff8bb1ae None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:35.874970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0ed4-52e7-4e8d-a4fc-bd4bdf7122b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:35.875613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0ed4-52e7-4e8d-a4fc-bd4bdf7122b7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:35.875892 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0ed4-52e7-4e8d-a4fc-bd4bdf7122b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:35.876162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0ed4-52e7-4e8d-a4fc-bd4bdf7122b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:35.943383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea0ed4-52e7-4e8d-a4fc-bd4bdf7122b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:35.944751 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea0ed4-52e7-4e8d-a4fc-bd4bdf7122b7 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:36.022863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:36.057685 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 610/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:01:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:01:36.064076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee32c3-5ddb-4bff-af3f-087d53f60030 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:36.132783 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:36.167892 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee32c3-5ddb-4bff-af3f-087d53f60030 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:36.201687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2e513-f178-453b-a049-f4688335b3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:36.202513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2e513-f178-453b-a049-f4688335b3ce None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:36.202795 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2e513-f178-453b-a049-f4688335b3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:36.203065 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2e513-f178-453b-a049-f4688335b3ce None admin] PATH_INFO: `/v3/projects/21f64426677041df90d9f8960f310de1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:36.204628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2e513-f178-453b-a049-f4688335b3ce None admin] RBAC: Authorizing `identity:get_project(project_id=21f64426677041df90d9f8960f310de1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:36.206076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2e513-f178-453b-a049-f4688335b3ce None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:36.207766 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 589/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jul 5 10:01:36 2022] GET /identity/v3/projects/21f64426677041df90d9f8960f310de1 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:36.216957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0ed4-52e7-4e8d-a4fc-bd4bdf7122b7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:01:36.347653 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0ed4-52e7-4e8d-a4fc-bd4bdf7122b7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:01:36.383813 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae14672-780d-4a65-ab2f-e84c793ce161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:01:36.384473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14672-780d-4a65-ab2f-e84c793ce161 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:01:36.384756 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14672-780d-4a65-ab2f-e84c793ce161 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:01:36.385022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14672-780d-4a65-ab2f-e84c793ce161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:01:36.385865 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14672-780d-4a65-ab2f-e84c793ce161 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=21f64426677041df90d9f8960f310de1, user.id=55dfc051d67643ea9db15c8fe807ba02) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:01:36.386172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14672-780d-4a65-ab2f-e84c793ce161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:01:36.387690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14672-780d-4a65-ab2f-e84c793ce161 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:01:36.394158 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 611/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:01:36 2022] GET /identity/v3/role_assignments?user.id=55dfc051d67643ea9db15c8fe807ba02&scope.project.id=21f64426677041df90d9f8960f310de1&role.id=c184a177010144429f495f3290d28221 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:01:46.196791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=33 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:01:46.391364 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=83 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:31.429321 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ba4c9-3442-4d0d-b5e5-89614ae2e4d5 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:31.429321 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ba4c9-3442-4d0d-b5e5-89614ae2e4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:31.429321 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ba4c9-3442-4d0d-b5e5-89614ae2e4d5 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:31.429321 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 590/1201] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:02:31.436113 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433f1b0-86cc-4253-8ec1-f8ad093dee9c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:31.436967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433f1b0-86cc-4253-8ec1-f8ad093dee9c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:31.437562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433f1b0-86cc-4253-8ec1-f8ad093dee9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:31.457706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-f433f1b0-86cc-4253-8ec1-f8ad093dee9c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:02:31.459036 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f433f1b0-86cc-4253-8ec1-f8ad093dee9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:02:31.551310 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 612/1202] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:02:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:02:31.576955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ba4c9-3442-4d0d-b5e5-89614ae2e4d5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:31.709176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ba4c9-3442-4d0d-b5e5-89614ae2e4d5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:31.758002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109bd910-5fa5-49f5-a7b0-9421511ba591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:31.758002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bd910-5fa5-49f5-a7b0-9421511ba591 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:31.758002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bd910-5fa5-49f5-a7b0-9421511ba591 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:31.758002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bd910-5fa5-49f5-a7b0-9421511ba591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:31.769106 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109bd910-5fa5-49f5-a7b0-9421511ba591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:31.770612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109bd910-5fa5-49f5-a7b0-9421511ba591 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-109bd910-5fa5-49f5-a7b0-9421511ba591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:31.788668 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:31.789769 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:31.790801 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:31.790801 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:31.790801 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:31.790801 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:31.798211 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 591/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:31.798458 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433f1b0-86cc-4253-8ec1-f8ad093dee9c None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:31.924763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433f1b0-86cc-4253-8ec1-f8ad093dee9c None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:31.959360 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:31.969320 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235720b-4b98-4cf2-a1a6-b98334296e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:31.970022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235720b-4b98-4cf2-a1a6-b98334296e16 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:31.970394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235720b-4b98-4cf2-a1a6-b98334296e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:31.970684 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235720b-4b98-4cf2-a1a6-b98334296e16 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:31.971601 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235720b-4b98-4cf2-a1a6-b98334296e16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:31.972093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235720b-4b98-4cf2-a1a6-b98334296e16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:31.973849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235720b-4b98-4cf2-a1a6-b98334296e16 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:31.990184 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 613/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:31.995970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109bd910-5fa5-49f5-a7b0-9421511ba591 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:32.067860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:32.131689 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109bd910-5fa5-49f5-a7b0-9421511ba591 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:32.175471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03366182-2299-4f74-8445-0e8b3ee90353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:32.176367 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03366182-2299-4f74-8445-0e8b3ee90353 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:32.176736 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03366182-2299-4f74-8445-0e8b3ee90353 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:32.177038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03366182-2299-4f74-8445-0e8b3ee90353 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:32.178015 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03366182-2299-4f74-8445-0e8b3ee90353 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:32.182806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03366182-2299-4f74-8445-0e8b3ee90353 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:32.214178 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 592/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jul 5 10:02:31 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:02:34.363642 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e9f73-687f-4ff2-af39-8bd80fb6e912 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:34.364255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e9f73-687f-4ff2-af39-8bd80fb6e912 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:34.364255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e9f73-687f-4ff2-af39-8bd80fb6e912 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:34.365742 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 614/1206] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:02:34.371070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83a475-1578-4dc5-9239-37170c9bd2d6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:34.371647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83a475-1578-4dc5-9239-37170c9bd2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:34.371647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83a475-1578-4dc5-9239-37170c9bd2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:34.389781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0e83a475-1578-4dc5-9239-37170c9bd2d6 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:02:34.391084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0e83a475-1578-4dc5-9239-37170c9bd2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:02:34.471243 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 593/1207] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:02:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:02:34.497822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e9f73-687f-4ff2-af39-8bd80fb6e912 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:34.619873 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e9f73-687f-4ff2-af39-8bd80fb6e912 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:34.655902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18656f0-f89f-491b-aaa9-fcb24b30c349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:34.657001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18656f0-f89f-491b-aaa9-fcb24b30c349 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:34.657383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18656f0-f89f-491b-aaa9-fcb24b30c349 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:34.657750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18656f0-f89f-491b-aaa9-fcb24b30c349 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:34.668604 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18656f0-f89f-491b-aaa9-fcb24b30c349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-e18656f0-f89f-491b-aaa9-fcb24b30c349 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:02:34.676730 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:34.682875 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:34.684045 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 615/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 10:02:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:34.684980 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83a475-1578-4dc5-9239-37170c9bd2d6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:34.727128 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:34.793640 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83a475-1578-4dc5-9239-37170c9bd2d6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:34.834952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e7ab2-60a1-4f39-a26e-b3fb0872f777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:34.835957 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e7ab2-60a1-4f39-a26e-b3fb0872f777 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:34.836267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e7ab2-60a1-4f39-a26e-b3fb0872f777 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:34.836532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e7ab2-60a1-4f39-a26e-b3fb0872f777 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:34.837279 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e7ab2-60a1-4f39-a26e-b3fb0872f777 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:34.837629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e7ab2-60a1-4f39-a26e-b3fb0872f777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:34.839100 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e7ab2-60a1-4f39-a26e-b3fb0872f777 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:34.852063 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 594/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 10:02:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:34.853401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18656f0-f89f-491b-aaa9-fcb24b30c349 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:34.919815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:34.972867 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18656f0-f89f-491b-aaa9-fcb24b30c349 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:35.014056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b8bc-640d-49f2-8dcf-2dc27e2840de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:35.014928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b8bc-640d-49f2-8dcf-2dc27e2840de None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:35.015244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b8bc-640d-49f2-8dcf-2dc27e2840de None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:35.015645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b8bc-640d-49f2-8dcf-2dc27e2840de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:35.028661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196b8bc-640d-49f2-8dcf-2dc27e2840de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:35.030329 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196b8bc-640d-49f2-8dcf-2dc27e2840de None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-1196b8bc-640d-49f2-8dcf-2dc27e2840de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:35.051025 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:35.052216 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:35.053163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:35.053163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:35.053163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:35.053163 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:35.053163 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 616/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:02:35.058781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e7ab2-60a1-4f39-a26e-b3fb0872f777 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:35.169552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:35.174249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e7ab2-60a1-4f39-a26e-b3fb0872f777 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:35.213362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda0ee8-b619-4f26-b026-570d21c125fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:35.214148 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda0ee8-b619-4f26-b026-570d21c125fd None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:35.214531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda0ee8-b619-4f26-b026-570d21c125fd None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:35.214881 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda0ee8-b619-4f26-b026-570d21c125fd None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:35.215692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda0ee8-b619-4f26-b026-570d21c125fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:35.216290 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda0ee8-b619-4f26-b026-570d21c125fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:35.217748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda0ee8-b619-4f26-b026-570d21c125fd None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:35.231844 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 595/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:35.238526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b8bc-640d-49f2-8dcf-2dc27e2840de None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:35.357337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b8bc-640d-49f2-8dcf-2dc27e2840de None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:35.359156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:35.406127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150590b-85a3-4028-9601-b0557b4e7b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:35.407167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150590b-85a3-4028-9601-b0557b4e7b69 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:35.407449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150590b-85a3-4028-9601-b0557b4e7b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:35.407725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150590b-85a3-4028-9601-b0557b4e7b69 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:35.419102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150590b-85a3-4028-9601-b0557b4e7b69 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:35.421134 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150590b-85a3-4028-9601-b0557b4e7b69 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-4150590b-85a3-4028-9601-b0557b4e7b69 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:02:35.441636 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:35.442904 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:35.444069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:35.444069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:35.444069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:02:35.444069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:02:35.444069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:02:35.444069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:02:35.444069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:02:35.444069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:02:35.444069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:35.444069 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:35.448044 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 617/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:35.449575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda0ee8-b619-4f26-b026-570d21c125fd None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:35.584797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda0ee8-b619-4f26-b026-570d21c125fd None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:35.626151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c8883-4da9-4dfb-898b-d67a88f6ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:35.626826 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c8883-4da9-4dfb-898b-d67a88f6ec1a None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:35.627109 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c8883-4da9-4dfb-898b-d67a88f6ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:35.627470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c8883-4da9-4dfb-898b-d67a88f6ec1a None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:35.628314 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c8883-4da9-4dfb-898b-d67a88f6ec1a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:35.628624 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c8883-4da9-4dfb-898b-d67a88f6ec1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:35.630061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c8883-4da9-4dfb-898b-d67a88f6ec1a None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:35.648855 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 596/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jul 5 10:02:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:02:35.657236 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150590b-85a3-4028-9601-b0557b4e7b69 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:35.775121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150590b-85a3-4028-9601-b0557b4e7b69 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:35.818706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4fe5b-34ca-437d-a281-0b1d5edbe499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:35.819671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4fe5b-34ca-437d-a281-0b1d5edbe499 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:35.819779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4fe5b-34ca-437d-a281-0b1d5edbe499 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:35.819991 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4fe5b-34ca-437d-a281-0b1d5edbe499 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:35.856145 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:35.907491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4fe5b-34ca-437d-a281-0b1d5edbe499 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:35.909990 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4fe5b-34ca-437d-a281-0b1d5edbe499 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:36.039465 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 618/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:02:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 10:02:36.044736 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c8883-4da9-4dfb-898b-d67a88f6ec1a None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:36.065310 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:36.150778 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c8883-4da9-4dfb-898b-d67a88f6ec1a None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:36.188049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c737da-0306-4ea4-b19d-7a55bd24a5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:36.188872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c737da-0306-4ea4-b19d-7a55bd24a5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:36.189162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c737da-0306-4ea4-b19d-7a55bd24a5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:36.190798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c737da-0306-4ea4-b19d-7a55bd24a5d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:36.204158 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c737da-0306-4ea4-b19d-7a55bd24a5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:36.206236 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c737da-0306-4ea4-b19d-7a55bd24a5d2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e4c737da-0306-4ea4-b19d-7a55bd24a5d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:36.230196 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:36.231961 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:36.233864 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:36.233864 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:36.233864 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:36.233864 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:36.233864 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 597/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:36.237907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4fe5b-34ca-437d-a281-0b1d5edbe499 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:36.308073 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:36.350563 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4fe5b-34ca-437d-a281-0b1d5edbe499 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:36.394247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52aaae-cd1d-4dca-9166-c4599a7efbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:36.395295 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52aaae-cd1d-4dca-9166-c4599a7efbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:36.395654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52aaae-cd1d-4dca-9166-c4599a7efbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:36.395937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52aaae-cd1d-4dca-9166-c4599a7efbc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:36.396789 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52aaae-cd1d-4dca-9166-c4599a7efbc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:36.397282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52aaae-cd1d-4dca-9166-c4599a7efbc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:36.398992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52aaae-cd1d-4dca-9166-c4599a7efbc8 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:36.418788 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 619/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:02:36.420051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c737da-0306-4ea4-b19d-7a55bd24a5d2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:36.496913 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:36.532976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c737da-0306-4ea4-b19d-7a55bd24a5d2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:36.577115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6f31c-097f-42ec-9d4f-a9deb1194326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:36.578077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6f31c-097f-42ec-9d4f-a9deb1194326 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:36.578430 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6f31c-097f-42ec-9d4f-a9deb1194326 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:36.579789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6f31c-097f-42ec-9d4f-a9deb1194326 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:36.587566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6f31c-097f-42ec-9d4f-a9deb1194326 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:36.589529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6f31c-097f-42ec-9d4f-a9deb1194326 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-24f6f31c-097f-42ec-9d4f-a9deb1194326 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:02:36.605346 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:36.606720 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:36.607766 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:02:36.607766 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:02:36.607766 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:02:36.607766 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:02:36.607766 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:02:36.607766 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:02:36.607766 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:36.607766 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:36.607766 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 598/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:02:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:02:36.613186 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52aaae-cd1d-4dca-9166-c4599a7efbc8 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:36.737302 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52aaae-cd1d-4dca-9166-c4599a7efbc8 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:36.767736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:36.782461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f650faf-dcc3-45d4-9130-1d8a77b538ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:36.783131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f650faf-dcc3-45d4-9130-1d8a77b538ae None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:36.783410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f650faf-dcc3-45d4-9130-1d8a77b538ae None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:36.783686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f650faf-dcc3-45d4-9130-1d8a77b538ae None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:36.784647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f650faf-dcc3-45d4-9130-1d8a77b538ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:36.784968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f650faf-dcc3-45d4-9130-1d8a77b538ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:36.786618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f650faf-dcc3-45d4-9130-1d8a77b538ae None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:36.799411 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 620/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:02:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:36.805414 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6f31c-097f-42ec-9d4f-a9deb1194326 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:36.925516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6f31c-097f-42ec-9d4f-a9deb1194326 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:36.956042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:36.962413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c03825-4b5c-4027-81c8-a4a938ba77c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:36.963175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c03825-4b5c-4027-81c8-a4a938ba77c9 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:36.963517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c03825-4b5c-4027-81c8-a4a938ba77c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:36.963855 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c03825-4b5c-4027-81c8-a4a938ba77c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:36.964746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c03825-4b5c-4027-81c8-a4a938ba77c9 None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=e9d4a50f842a48b1875900426c6a1ac5) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:36.965126 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c03825-4b5c-4027-81c8-a4a938ba77c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:36.966655 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c03825-4b5c-4027-81c8-a4a938ba77c9 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:36.972905 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 599/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:02:36 2022] GET /identity/v3/role_assignments?user.id=e9d4a50f842a48b1875900426c6a1ac5&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:39.028875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d033e3-549e-4b4f-90df-9d22b2a415fc None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:39.029403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d033e3-549e-4b4f-90df-9d22b2a415fc None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:39.029403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d033e3-549e-4b4f-90df-9d22b2a415fc None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:39.030791 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 621/1220] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:02:39.037402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010d63a-7d12-4916-9299-c9547309ea34 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:39.037892 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010d63a-7d12-4916-9299-c9547309ea34 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:39.038408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010d63a-7d12-4916-9299-c9547309ea34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:39.057516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f010d63a-7d12-4916-9299-c9547309ea34 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:02:39.058742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f010d63a-7d12-4916-9299-c9547309ea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:02:39.128454 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 600/1221] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:02:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:02:39.150881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d033e3-549e-4b4f-90df-9d22b2a415fc None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:39.264495 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d033e3-549e-4b4f-90df-9d22b2a415fc None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:39.301187 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb0395-9b4c-4f02-99f7-beb9542df36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:39.301997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb0395-9b4c-4f02-99f7-beb9542df36c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:39.302307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb0395-9b4c-4f02-99f7-beb9542df36c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:39.302579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb0395-9b4c-4f02-99f7-beb9542df36c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:39.314059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb0395-9b4c-4f02-99f7-beb9542df36c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-93eb0395-9b4c-4f02-99f7-beb9542df36c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:02:39.322587 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:39.324059 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:39.325014 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 622/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 10:02:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:39.330525 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010d63a-7d12-4916-9299-c9547309ea34 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:39.437105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010d63a-7d12-4916-9299-c9547309ea34 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:39.461848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:39.473680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa28eca-d7f4-43a5-97d2-05332cb3e6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:39.474511 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa28eca-d7f4-43a5-97d2-05332cb3e6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:39.474798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa28eca-d7f4-43a5-97d2-05332cb3e6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:39.475091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa28eca-d7f4-43a5-97d2-05332cb3e6e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:39.475857 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa28eca-d7f4-43a5-97d2-05332cb3e6e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:39.476232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa28eca-d7f4-43a5-97d2-05332cb3e6e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:39.477806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa28eca-d7f4-43a5-97d2-05332cb3e6e2 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:39.488033 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 601/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 10:02:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:39.494650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb0395-9b4c-4f02-99f7-beb9542df36c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:39.543038 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:39.607919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb0395-9b4c-4f02-99f7-beb9542df36c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:39.642149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016e654-bd73-443c-b457-c5317cf2f728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:39.642976 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016e654-bd73-443c-b457-c5317cf2f728 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:39.643257 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016e654-bd73-443c-b457-c5317cf2f728 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:39.643531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016e654-bd73-443c-b457-c5317cf2f728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:39.654163 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016e654-bd73-443c-b457-c5317cf2f728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:39.655666 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016e654-bd73-443c-b457-c5317cf2f728 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-c016e654-bd73-443c-b457-c5317cf2f728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:39.673049 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:39.674139 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:39.675202 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:39.675202 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:39.675202 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:39.675202 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:39.675202 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 623/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:39.680692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa28eca-d7f4-43a5-97d2-05332cb3e6e2 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:39.790630 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa28eca-d7f4-43a5-97d2-05332cb3e6e2 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:39.827054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00adcb4-35b8-4adc-bd26-16de4aacbe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:39.827760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00adcb4-35b8-4adc-bd26-16de4aacbe05 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:39.828167 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00adcb4-35b8-4adc-bd26-16de4aacbe05 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:39.828531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00adcb4-35b8-4adc-bd26-16de4aacbe05 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:39.829264 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00adcb4-35b8-4adc-bd26-16de4aacbe05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:39.829601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00adcb4-35b8-4adc-bd26-16de4aacbe05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:39.831289 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00adcb4-35b8-4adc-bd26-16de4aacbe05 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:39.844390 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 602/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:39.850391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016e654-bd73-443c-b457-c5317cf2f728 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:39.947908 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:39.956101 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016e654-bd73-443c-b457-c5317cf2f728 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:39.991221 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff286b4-546a-4342-8d84-c8a70d087579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:39.992108 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff286b4-546a-4342-8d84-c8a70d087579 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:39.992418 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff286b4-546a-4342-8d84-c8a70d087579 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:39.992783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff286b4-546a-4342-8d84-c8a70d087579 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:40.001500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff286b4-546a-4342-8d84-c8a70d087579 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:40.003336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff286b4-546a-4342-8d84-c8a70d087579 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-3ff286b4-546a-4342-8d84-c8a70d087579 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:02:40.018372 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:40.019578 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:40.021137 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:40.021137 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:40.021137 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:02:40.021137 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:02:40.021137 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:02:40.021137 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:02:40.021137 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:02:40.021137 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:02:40.021137 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:40.021137 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:40.021137 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 624/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:40.026467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00adcb4-35b8-4adc-bd26-16de4aacbe05 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:40.077382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:40.133061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00adcb4-35b8-4adc-bd26-16de4aacbe05 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:40.167969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16e77-4fb2-4a73-aee0-4ceb8f98bf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:40.169966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16e77-4fb2-4a73-aee0-4ceb8f98bf74 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:40.169966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16e77-4fb2-4a73-aee0-4ceb8f98bf74 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:40.169966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb16e77-4fb2-4a73-aee0-4ceb8f98bf74 None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:40.170376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16e77-4fb2-4a73-aee0-4ceb8f98bf74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:40.170376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16e77-4fb2-4a73-aee0-4ceb8f98bf74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:40.171757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb16e77-4fb2-4a73-aee0-4ceb8f98bf74 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:40.189947 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 603/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jul 5 10:02:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:40.195856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff286b4-546a-4342-8d84-c8a70d087579 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:40.302496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff286b4-546a-4342-8d84-c8a70d087579 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:40.338470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb57a8c-d0a9-483f-8867-bc549192660c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:40.339286 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57a8c-d0a9-483f-8867-bc549192660c None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:40.339572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57a8c-d0a9-483f-8867-bc549192660c None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:40.339848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57a8c-d0a9-483f-8867-bc549192660c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:40.350717 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb57a8c-d0a9-483f-8867-bc549192660c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:40.352630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb57a8c-d0a9-483f-8867-bc549192660c None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-beb57a8c-d0a9-483f-8867-bc549192660c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:40.370851 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:40.371945 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:40.373097 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:40.373097 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:40.373097 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:40.373097 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:40.373097 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 625/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:40.377976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16e77-4fb2-4a73-aee0-4ceb8f98bf74 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:40.483639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb16e77-4fb2-4a73-aee0-4ceb8f98bf74 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:40.485985 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:40.520981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08c9b0-7f99-4efa-80bd-0dcdcf04de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:40.521654 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c9b0-7f99-4efa-80bd-0dcdcf04de4e None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:40.521935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c9b0-7f99-4efa-80bd-0dcdcf04de4e None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:40.522209 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c9b0-7f99-4efa-80bd-0dcdcf04de4e None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:40.522960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08c9b0-7f99-4efa-80bd-0dcdcf04de4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:40.523266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08c9b0-7f99-4efa-80bd-0dcdcf04de4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:40.524639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08c9b0-7f99-4efa-80bd-0dcdcf04de4e None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:40.536525 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 604/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:40.542694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb57a8c-d0a9-483f-8867-bc549192660c None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:40.552816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:40.657559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb57a8c-d0a9-483f-8867-bc549192660c None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:40.693848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d47f87-fce3-4a55-9fd7-a419cd497951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:40.694964 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d47f87-fce3-4a55-9fd7-a419cd497951 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:40.695334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d47f87-fce3-4a55-9fd7-a419cd497951 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:40.695669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d47f87-fce3-4a55-9fd7-a419cd497951 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:40.705867 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d47f87-fce3-4a55-9fd7-a419cd497951 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:40.707398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d47f87-fce3-4a55-9fd7-a419cd497951 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-90d47f87-fce3-4a55-9fd7-a419cd497951 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:02:40.721952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:40.723093 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:40.724178 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:02:40.724178 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:02:40.724178 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:02:40.724178 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:02:40.724178 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:02:40.724178 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:02:40.724178 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:40.724178 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:40.724178 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 626/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:02:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:40.729174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08c9b0-7f99-4efa-80bd-0dcdcf04de4e None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:40.838056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08c9b0-7f99-4efa-80bd-0dcdcf04de4e None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:40.874733 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2e622-6739-4cf6-96c8-839babd1cdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:40.875536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2e622-6739-4cf6-96c8-839babd1cdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:40.875934 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2e622-6739-4cf6-96c8-839babd1cdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:40.876238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2e622-6739-4cf6-96c8-839babd1cdf4 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:40.877355 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2e622-6739-4cf6-96c8-839babd1cdf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:40.877740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2e622-6739-4cf6-96c8-839babd1cdf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:40.879587 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2e622-6739-4cf6-96c8-839babd1cdf4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:40.894856 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 605/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:02:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:02:40.901189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d47f87-fce3-4a55-9fd7-a419cd497951 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:40.967556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:41.004592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d47f87-fce3-4a55-9fd7-a419cd497951 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:41.039616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7c703-e629-43bf-b025-26b7f64de320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:41.040568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7c703-e629-43bf-b025-26b7f64de320 None admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:41.040846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7c703-e629-43bf-b025-26b7f64de320 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:41.041132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7c703-e629-43bf-b025-26b7f64de320 None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/e9d4a50f842a48b1875900426c6a1ac5/roles/1821d382b71242ff8f94e4930b6ab71d` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:41.057999 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7c703-e629-43bf-b025-26b7f64de320 None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=e9d4a50f842a48b1875900426c6a1ac5, role_id=1821d382b71242ff8f94e4930b6ab71d)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:41.060063 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7c703-e629-43bf-b025-26b7f64de320 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:41.075367 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 627/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 10:02:40 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/e9d4a50f842a48b1875900426c6a1ac5/roles/1821d382b71242ff8f94e4930b6ab71d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:02:43.114418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c399f-7e37-40d7-8696-5a899adf0ed8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:43.114940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c399f-7e37-40d7-8696-5a899adf0ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:43.114940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c399f-7e37-40d7-8696-5a899adf0ed8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:43.116314 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 606/1233] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:02:43.121857 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf623fd-a252-46a6-afab-32fe00f0bbfd None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:43.122304 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf623fd-a252-46a6-afab-32fe00f0bbfd None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:43.122439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf623fd-a252-46a6-afab-32fe00f0bbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:43.139316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-2cf623fd-a252-46a6-afab-32fe00f0bbfd None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:02:43.140506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2cf623fd-a252-46a6-afab-32fe00f0bbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:02:43.212053 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 628/1234] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:02:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:02:43.235950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c399f-7e37-40d7-8696-5a899adf0ed8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:43.261496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:43.339572 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c399f-7e37-40d7-8696-5a899adf0ed8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:43.375955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79af3d-79cb-4250-8c6d-7f627a9b4991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:43.376710 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79af3d-79cb-4250-8c6d-7f627a9b4991 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:43.376993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79af3d-79cb-4250-8c6d-7f627a9b4991 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:43.377266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79af3d-79cb-4250-8c6d-7f627a9b4991 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:43.388186 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79af3d-79cb-4250-8c6d-7f627a9b4991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-cf79af3d-79cb-4250-8c6d-7f627a9b4991 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:02:43.395997 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:43.397041 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:43.398455 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 607/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jul 5 10:02:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:43.403468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf623fd-a252-46a6-afab-32fe00f0bbfd None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:43.450079 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:43.505963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf623fd-a252-46a6-afab-32fe00f0bbfd None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:43.541458 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38863feb-a022-4d51-9507-2055d24bb748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:43.542136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38863feb-a022-4d51-9507-2055d24bb748 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:43.542512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38863feb-a022-4d51-9507-2055d24bb748 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:43.542787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38863feb-a022-4d51-9507-2055d24bb748 None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:43.543532 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38863feb-a022-4d51-9507-2055d24bb748 None admin] RBAC: Adding query filter params (name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:43.543843 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38863feb-a022-4d51-9507-2055d24bb748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:43.545347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38863feb-a022-4d51-9507-2055d24bb748 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:43.554391 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 629/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jul 5 10:02:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:43.561217 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79af3d-79cb-4250-8c6d-7f627a9b4991 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:43.659069 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:43.677003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79af3d-79cb-4250-8c6d-7f627a9b4991 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:43.713436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778acd69-16e5-4e0e-b440-0efc69c5c132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:43.714354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778acd69-16e5-4e0e-b440-0efc69c5c132 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:43.714648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778acd69-16e5-4e0e-b440-0efc69c5c132 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:43.714992 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778acd69-16e5-4e0e-b440-0efc69c5c132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:43.727188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778acd69-16e5-4e0e-b440-0efc69c5c132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:43.728971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778acd69-16e5-4e0e-b440-0efc69c5c132 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-778acd69-16e5-4e0e-b440-0efc69c5c132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:43.747377 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:43.748415 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:43.749400 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:43.749400 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:43.749400 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:43.749400 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:43.749400 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 608/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:43.754717 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38863feb-a022-4d51-9507-2055d24bb748 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:43.870902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38863feb-a022-4d51-9507-2055d24bb748 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:43.880676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:43.917497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05eb44-f100-480b-b2e1-65e84ada77d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:43.918159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05eb44-f100-480b-b2e1-65e84ada77d9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:43.918530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05eb44-f100-480b-b2e1-65e84ada77d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:43.918809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05eb44-f100-480b-b2e1-65e84ada77d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:43.919534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05eb44-f100-480b-b2e1-65e84ada77d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:43.919841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05eb44-f100-480b-b2e1-65e84ada77d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:43.921305 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05eb44-f100-480b-b2e1-65e84ada77d9 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:43.935011 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 630/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:43.942331 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778acd69-16e5-4e0e-b440-0efc69c5c132 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:44.052231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778acd69-16e5-4e0e-b440-0efc69c5c132 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:44.089526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f00674-f146-4d28-a2d3-5d50c8cb8cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:44.090443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f00674-f146-4d28-a2d3-5d50c8cb8cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:44.090831 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f00674-f146-4d28-a2d3-5d50c8cb8cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:44.091157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f00674-f146-4d28-a2d3-5d50c8cb8cc4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:44.094695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:44.100090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f00674-f146-4d28-a2d3-5d50c8cb8cc4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:44.101748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f00674-f146-4d28-a2d3-5d50c8cb8cc4 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-65f00674-f146-4d28-a2d3-5d50c8cb8cc4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:02:44.115982 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:44.117617 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:44.118715 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:44.118715 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:44.118715 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:02:44.118715 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:02:44.118715 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:02:44.118715 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:02:44.118715 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:02:44.118715 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:02:44.118715 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:44.118715 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:44.118715 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 609/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:44.123397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05eb44-f100-480b-b2e1-65e84ada77d9 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:44.224154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05eb44-f100-480b-b2e1-65e84ada77d9 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:44.260123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c46390-756a-4812-9cd3-8ecf000c0ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:44.260812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c46390-756a-4812-9cd3-8ecf000c0ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:44.261109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c46390-756a-4812-9cd3-8ecf000c0ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:44.261400 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c46390-756a-4812-9cd3-8ecf000c0ae0 None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:44.262306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c46390-756a-4812-9cd3-8ecf000c0ae0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:44.262613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c46390-756a-4812-9cd3-8ecf000c0ae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:44.264198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c46390-756a-4812-9cd3-8ecf000c0ae0 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:44.282192 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 631/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jul 5 10:02:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:02:44.288634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f00674-f146-4d28-a2d3-5d50c8cb8cc4 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:44.403540 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f00674-f146-4d28-a2d3-5d50c8cb8cc4 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:44.440464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854663d-e543-4ccb-b5bb-ccfa6e1934cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:44.441180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854663d-e543-4ccb-b5bb-ccfa6e1934cf None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:44.441512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854663d-e543-4ccb-b5bb-ccfa6e1934cf None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:44.441789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854663d-e543-4ccb-b5bb-ccfa6e1934cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:44.506210 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:44.514919 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854663d-e543-4ccb-b5bb-ccfa6e1934cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:44.516529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854663d-e543-4ccb-b5bb-ccfa6e1934cf None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:44.641220 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 610/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:02:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:02:44.647616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c46390-756a-4812-9cd3-8ecf000c0ae0 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:44.677375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:44.752181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c46390-756a-4812-9cd3-8ecf000c0ae0 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:44.786803 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28512d32-733a-4c7e-9a10-03dd56268419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:44.787606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28512d32-733a-4c7e-9a10-03dd56268419 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:44.787884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28512d32-733a-4c7e-9a10-03dd56268419 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:44.788241 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28512d32-733a-4c7e-9a10-03dd56268419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:44.807833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28512d32-733a-4c7e-9a10-03dd56268419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:44.809309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28512d32-733a-4c7e-9a10-03dd56268419 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-28512d32-733a-4c7e-9a10-03dd56268419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:44.827705 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:44.828773 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:44.829713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:44.829713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:44.829713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:44.829713 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:44.829713 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 632/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:44.835014 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854663d-e543-4ccb-b5bb-ccfa6e1934cf None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:44.939319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:44.945522 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854663d-e543-4ccb-b5bb-ccfa6e1934cf None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:44.987638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33efef5-23ab-4627-b75d-8eb8b4ef3c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:44.988493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33efef5-23ab-4627-b75d-8eb8b4ef3c96 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:44.988888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33efef5-23ab-4627-b75d-8eb8b4ef3c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:44.989167 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33efef5-23ab-4627-b75d-8eb8b4ef3c96 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:44.990193 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33efef5-23ab-4627-b75d-8eb8b4ef3c96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:44.990536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33efef5-23ab-4627-b75d-8eb8b4ef3c96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:44.992007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33efef5-23ab-4627-b75d-8eb8b4ef3c96 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:45.006722 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 611/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:45.012749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28512d32-733a-4c7e-9a10-03dd56268419 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:45.098215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:45.123176 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28512d32-733a-4c7e-9a10-03dd56268419 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:45.157513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e6719-e5bb-4a52-b4d2-9e143ff55141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:45.158417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e6719-e5bb-4a52-b4d2-9e143ff55141 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:45.158705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e6719-e5bb-4a52-b4d2-9e143ff55141 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:45.158988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e6719-e5bb-4a52-b4d2-9e143ff55141 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:45.166811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e6719-e5bb-4a52-b4d2-9e143ff55141 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:45.168595 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e6719-e5bb-4a52-b4d2-9e143ff55141 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-b56e6719-e5bb-4a52-b4d2-9e143ff55141 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:02:45.184010 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:45.185188 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:45.186541 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:02:45.186541 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:02:45.186541 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:02:45.186541 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:02:45.186541 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:02:45.186541 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:02:45.186541 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:45.186541 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:45.186541 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 633/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:45.191369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33efef5-23ab-4627-b75d-8eb8b4ef3c96 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:45.307113 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33efef5-23ab-4627-b75d-8eb8b4ef3c96 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:45.345352 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5d558-4108-49ed-ab8a-cb47bdf3d0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:45.346290 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5d558-4108-49ed-ab8a-cb47bdf3d0ce None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:45.346838 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5d558-4108-49ed-ab8a-cb47bdf3d0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:45.347213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5d558-4108-49ed-ab8a-cb47bdf3d0ce None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:45.348320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5d558-4108-49ed-ab8a-cb47bdf3d0ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:45.348713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5d558-4108-49ed-ab8a-cb47bdf3d0ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:45.350733 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5d558-4108-49ed-ab8a-cb47bdf3d0ce None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:45.364569 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 612/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:02:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:45.371065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e6719-e5bb-4a52-b4d2-9e143ff55141 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:45.476789 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e6719-e5bb-4a52-b4d2-9e143ff55141 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:45.511824 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a6348-5dd3-4038-9010-3ad37e96025b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:45.512493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a6348-5dd3-4038-9010-3ad37e96025b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:45.512779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a6348-5dd3-4038-9010-3ad37e96025b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:45.513047 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a6348-5dd3-4038-9010-3ad37e96025b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:45.513883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a6348-5dd3-4038-9010-3ad37e96025b None admin] RBAC: Adding query filter params (role.id=1821d382b71242ff8f94e4930b6ab71d, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=e9d4a50f842a48b1875900426c6a1ac5) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:45.514211 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a6348-5dd3-4038-9010-3ad37e96025b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:45.515654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a6348-5dd3-4038-9010-3ad37e96025b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:45.522602 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 634/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:02:45 2022] GET /identity/v3/role_assignments?user.id=e9d4a50f842a48b1875900426c6a1ac5&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=1821d382b71242ff8f94e4930b6ab71d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:47.914353 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f387cc-53ad-4a2c-bef6-42b3cd88ba3e None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:47.914916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f387cc-53ad-4a2c-bef6-42b3cd88ba3e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:47.914916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f387cc-53ad-4a2c-bef6-42b3cd88ba3e None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:47.916415 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 613/1247] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:02:47.922540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dbc42-4c00-424c-908a-3beaa7a0a814 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:47.923075 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dbc42-4c00-424c-908a-3beaa7a0a814 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:47.923075 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dbc42-4c00-424c-908a-3beaa7a0a814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:47.928918 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:47.943725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-849dbc42-4c00-424c-908a-3beaa7a0a814 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:02:47.944951 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-849dbc42-4c00-424c-908a-3beaa7a0a814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:02:48.013909 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 635/1248] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:02:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:02:48.037805 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f387cc-53ad-4a2c-bef6-42b3cd88ba3e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:48.056630 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:48.147754 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f387cc-53ad-4a2c-bef6-42b3cd88ba3e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:48.186844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca6203-07ff-47a5-8374-b4fa16622d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:48.187717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca6203-07ff-47a5-8374-b4fa16622d72 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:48.187967 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca6203-07ff-47a5-8374-b4fa16622d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:48.188221 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca6203-07ff-47a5-8374-b4fa16622d72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:48.199773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca6203-07ff-47a5-8374-b4fa16622d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-4bca6203-07ff-47a5-8374-b4fa16622d72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:02:48.208169 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:48.209243 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:48.210153 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 614/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:02:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:48.216173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dbc42-4c00-424c-908a-3beaa7a0a814 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:48.332154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dbc42-4c00-424c-908a-3beaa7a0a814 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:48.365726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:48.371212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c795e0-fa57-45c5-ad79-3b49a343355f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:48.371895 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c795e0-fa57-45c5-ad79-3b49a343355f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:48.372169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c795e0-fa57-45c5-ad79-3b49a343355f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:48.372443 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c795e0-fa57-45c5-ad79-3b49a343355f None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:48.373184 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c795e0-fa57-45c5-ad79-3b49a343355f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:48.373487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c795e0-fa57-45c5-ad79-3b49a343355f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:48.374981 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c795e0-fa57-45c5-ad79-3b49a343355f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:48.391237 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 636/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:02:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:48.394346 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca6203-07ff-47a5-8374-b4fa16622d72 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:48.487672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:48.513854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca6203-07ff-47a5-8374-b4fa16622d72 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:48.554291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41349dd5-3815-4102-bd8b-bfbdaa4fc8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:48.555322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41349dd5-3815-4102-bd8b-bfbdaa4fc8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:48.555813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41349dd5-3815-4102-bd8b-bfbdaa4fc8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:48.556111 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41349dd5-3815-4102-bd8b-bfbdaa4fc8f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:48.569189 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41349dd5-3815-4102-bd8b-bfbdaa4fc8f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:48.570911 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41349dd5-3815-4102-bd8b-bfbdaa4fc8f1 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-41349dd5-3815-4102-bd8b-bfbdaa4fc8f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:48.591863 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:48.593026 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:48.594056 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:48.594056 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:48.594056 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:48.594056 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:48.594056 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 615/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:48.599679 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c795e0-fa57-45c5-ad79-3b49a343355f None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:48.725579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c795e0-fa57-45c5-ad79-3b49a343355f None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:48.770492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a30d59-7109-4ea5-9c81-bb07e0217d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:48.770492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a30d59-7109-4ea5-9c81-bb07e0217d8e None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:48.770826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a30d59-7109-4ea5-9c81-bb07e0217d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:48.771969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a30d59-7109-4ea5-9c81-bb07e0217d8e None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:48.771969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a30d59-7109-4ea5-9c81-bb07e0217d8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:48.772150 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a30d59-7109-4ea5-9c81-bb07e0217d8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:48.774363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a30d59-7109-4ea5-9c81-bb07e0217d8e None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:48.786025 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 637/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:02:48.791923 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41349dd5-3815-4102-bd8b-bfbdaa4fc8f1 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:48.897241 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41349dd5-3815-4102-bd8b-bfbdaa4fc8f1 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:48.926386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:48.938257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eef1e5-5c16-4f3b-964d-cafc6270f193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:48.938257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eef1e5-5c16-4f3b-964d-cafc6270f193 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:48.938257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eef1e5-5c16-4f3b-964d-cafc6270f193 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:48.938257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eef1e5-5c16-4f3b-964d-cafc6270f193 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:48.946395 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eef1e5-5c16-4f3b-964d-cafc6270f193 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:48.947910 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eef1e5-5c16-4f3b-964d-cafc6270f193 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-d1eef1e5-5c16-4f3b-964d-cafc6270f193 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:02:48.965547 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:48.966623 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:48.967885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:48.967885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:48.967885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:02:48.967885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:02:48.967885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:02:48.967885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:02:48.967885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:02:48.967885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:02:48.967885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:48.967885 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:48.967885 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 616/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:02:48.973009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a30d59-7109-4ea5-9c81-bb07e0217d8e None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:49.032351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:49.081398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a30d59-7109-4ea5-9c81-bb07e0217d8e None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:49.119907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8463bf-6dd2-4cf1-9567-1a43df74646b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:49.120694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8463bf-6dd2-4cf1-9567-1a43df74646b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:49.121031 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8463bf-6dd2-4cf1-9567-1a43df74646b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:49.121323 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8463bf-6dd2-4cf1-9567-1a43df74646b None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:49.122240 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8463bf-6dd2-4cf1-9567-1a43df74646b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:49.122562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8463bf-6dd2-4cf1-9567-1a43df74646b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:49.124407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8463bf-6dd2-4cf1-9567-1a43df74646b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:49.143436 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 638/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jul 5 10:02:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:49.149489 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eef1e5-5c16-4f3b-964d-cafc6270f193 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:49.262175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eef1e5-5c16-4f3b-964d-cafc6270f193 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:49.297692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51776262-a6d1-4b31-bb31-ff310eea090c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:49.298531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51776262-a6d1-4b31-bb31-ff310eea090c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:49.298760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51776262-a6d1-4b31-bb31-ff310eea090c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:49.299007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51776262-a6d1-4b31-bb31-ff310eea090c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:49.363493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:49.376926 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51776262-a6d1-4b31-bb31-ff310eea090c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:49.378413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51776262-a6d1-4b31-bb31-ff310eea090c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:49.490215 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 617/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:02:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:02:49.496378 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8463bf-6dd2-4cf1-9567-1a43df74646b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:49.614743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8463bf-6dd2-4cf1-9567-1a43df74646b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:49.629778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:49.656914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09652323-2045-45c3-8ea9-82f668a6dff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:49.657805 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652323-2045-45c3-8ea9-82f668a6dff7 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:49.658124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652323-2045-45c3-8ea9-82f668a6dff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:49.658582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652323-2045-45c3-8ea9-82f668a6dff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:49.670909 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09652323-2045-45c3-8ea9-82f668a6dff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:49.672903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09652323-2045-45c3-8ea9-82f668a6dff7 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-09652323-2045-45c3-8ea9-82f668a6dff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:49.692298 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:49.693405 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:49.694776 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:49.694776 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:49.694776 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:49.694776 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:49.694776 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 639/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:49.699553 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51776262-a6d1-4b31-bb31-ff310eea090c None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:49.791510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:49.804542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51776262-a6d1-4b31-bb31-ff310eea090c None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:49.848769 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a5df1-4678-456d-a1e8-11f4107c95f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:49.849544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a5df1-4678-456d-a1e8-11f4107c95f7 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:49.849827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a5df1-4678-456d-a1e8-11f4107c95f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:49.850101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a5df1-4678-456d-a1e8-11f4107c95f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:49.850903 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a5df1-4678-456d-a1e8-11f4107c95f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:49.851207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a5df1-4678-456d-a1e8-11f4107c95f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:49.852794 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a5df1-4678-456d-a1e8-11f4107c95f7 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:49.864769 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 618/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:49.870758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09652323-2045-45c3-8ea9-82f668a6dff7 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:49.976381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09652323-2045-45c3-8ea9-82f668a6dff7 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:50.012890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aae0b-a48e-4eba-b469-41790006da69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:50.013683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aae0b-a48e-4eba-b469-41790006da69 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:50.014003 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aae0b-a48e-4eba-b469-41790006da69 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:50.014446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aae0b-a48e-4eba-b469-41790006da69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:50.021894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aae0b-a48e-4eba-b469-41790006da69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:50.023442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aae0b-a48e-4eba-b469-41790006da69 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-b30aae0b-a48e-4eba-b469-41790006da69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:02:50.038901 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:50.040088 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:50.041386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:02:50.041386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:02:50.041386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:02:50.041386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:02:50.041386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:02:50.041386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:02:50.041386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:50.041386 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:50.041386 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 640/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:02:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:02:50.046353 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a5df1-4678-456d-a1e8-11f4107c95f7 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:50.155326 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a5df1-4678-456d-a1e8-11f4107c95f7 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:50.190004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ea147-b696-4c08-96a0-a366da954017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:50.190711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ea147-b696-4c08-96a0-a366da954017 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:50.190994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ea147-b696-4c08-96a0-a366da954017 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:50.191340 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ea147-b696-4c08-96a0-a366da954017 None admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:50.192115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ea147-b696-4c08-96a0-a366da954017 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:50.192429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ea147-b696-4c08-96a0-a366da954017 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:50.193875 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ea147-b696-4c08-96a0-a366da954017 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:50.211390 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 619/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:02:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:02:50.212752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aae0b-a48e-4eba-b469-41790006da69 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:50.241486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:50.326776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aae0b-a48e-4eba-b469-41790006da69 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:50.365196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63b9e0-22a8-4de6-a484-9bf6ae6928a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:50.365968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63b9e0-22a8-4de6-a484-9bf6ae6928a9 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:50.368526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63b9e0-22a8-4de6-a484-9bf6ae6928a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:50.368526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63b9e0-22a8-4de6-a484-9bf6ae6928a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:50.368708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63b9e0-22a8-4de6-a484-9bf6ae6928a9 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=e9d4a50f842a48b1875900426c6a1ac5) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:50.369192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63b9e0-22a8-4de6-a484-9bf6ae6928a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:50.375948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63b9e0-22a8-4de6-a484-9bf6ae6928a9 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:50.376954 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 641/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:02:50 2022] GET /identity/v3/role_assignments?user.id=e9d4a50f842a48b1875900426c6a1ac5&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=c184a177010144429f495f3290d28221 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:52.395536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01153f75-60d8-41fe-b5ed-0a3274255ef6 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:52.396098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01153f75-60d8-41fe-b5ed-0a3274255ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:52.396098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01153f75-60d8-41fe-b5ed-0a3274255ef6 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:52.397472 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 620/1261] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:02:52.403161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b9333-8281-40c6-9862-1624ecb2c74f None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:52.403635 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b9333-8281-40c6-9862-1624ecb2c74f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:52.403747 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b9333-8281-40c6-9862-1624ecb2c74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:52.431618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-d07b9333-8281-40c6-9862-1624ecb2c74f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:02:52.432851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d07b9333-8281-40c6-9862-1624ecb2c74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:02:52.502842 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 642/1262] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:02:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:02:52.528244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01153f75-60d8-41fe-b5ed-0a3274255ef6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:52.543336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:52.648048 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01153f75-60d8-41fe-b5ed-0a3274255ef6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:52.682970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1e56b-a570-4ad3-9554-66bee11ec6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:52.683764 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1e56b-a570-4ad3-9554-66bee11ec6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:52.684050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1e56b-a570-4ad3-9554-66bee11ec6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:52.684331 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1e56b-a570-4ad3-9554-66bee11ec6f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:52.695175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1e56b-a570-4ad3-9554-66bee11ec6f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-def1e56b-a570-4ad3-9554-66bee11ec6f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:02:52.703161 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:52.704217 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:52.705361 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 621/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:02:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:52.712821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b9333-8281-40c6-9862-1624ecb2c74f None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:52.732979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:52.820983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b9333-8281-40c6-9862-1624ecb2c74f None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:52.856091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f26c15-4bd7-4f82-9a85-63dffa04a94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:52.856750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f26c15-4bd7-4f82-9a85-63dffa04a94f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:52.857029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f26c15-4bd7-4f82-9a85-63dffa04a94f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:52.857300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f26c15-4bd7-4f82-9a85-63dffa04a94f None admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:52.858041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f26c15-4bd7-4f82-9a85-63dffa04a94f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:52.858375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f26c15-4bd7-4f82-9a85-63dffa04a94f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:52.859746 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f26c15-4bd7-4f82-9a85-63dffa04a94f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:52.868177 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 643/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:02:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:52.876819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1e56b-a570-4ad3-9554-66bee11ec6f5 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:52.951082 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:52.990429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1e56b-a570-4ad3-9554-66bee11ec6f5 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:53.024671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28109d-abb4-453c-b0f7-ab9255925dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:53.025453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28109d-abb4-453c-b0f7-ab9255925dbe None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:53.025735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28109d-abb4-453c-b0f7-ab9255925dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:53.026005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28109d-abb4-453c-b0f7-ab9255925dbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:53.036686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28109d-abb4-453c-b0f7-ab9255925dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:53.038136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28109d-abb4-453c-b0f7-ab9255925dbe None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-9b28109d-abb4-453c-b0f7-ab9255925dbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:53.055284 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:53.056207 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:53.057152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:53.057152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:53.057152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:53.057152 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:53.057152 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 622/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:53.065197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f26c15-4bd7-4f82-9a85-63dffa04a94f None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:53.160273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:53.179767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f26c15-4bd7-4f82-9a85-63dffa04a94f None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:53.215152 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131ef4f-5f3a-43f9-a770-bd90532db1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:53.216133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131ef4f-5f3a-43f9-a770-bd90532db1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:53.216565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131ef4f-5f3a-43f9-a770-bd90532db1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:53.216993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131ef4f-5f3a-43f9-a770-bd90532db1e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:53.218157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131ef4f-5f3a-43f9-a770-bd90532db1e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:53.218637 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131ef4f-5f3a-43f9-a770-bd90532db1e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:53.220843 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131ef4f-5f3a-43f9-a770-bd90532db1e5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:53.240528 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 644/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:53.247505 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28109d-abb4-453c-b0f7-ab9255925dbe None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:53.350872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28109d-abb4-453c-b0f7-ab9255925dbe None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:53.370504 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:53.386013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dde216-6632-414d-95cd-323c3aa0df7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:53.386844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde216-6632-414d-95cd-323c3aa0df7f None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:53.387137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde216-6632-414d-95cd-323c3aa0df7f None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:53.387411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde216-6632-414d-95cd-323c3aa0df7f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:53.395294 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dde216-6632-414d-95cd-323c3aa0df7f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:53.396735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dde216-6632-414d-95cd-323c3aa0df7f None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-17dde216-6632-414d-95cd-323c3aa0df7f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:02:53.418254 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:53.419367 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:53.422178 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:53.422178 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:53.422178 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:02:53.422178 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:02:53.422178 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:02:53.422178 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:02:53.422178 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:02:53.422178 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:02:53.422178 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:53.422178 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:53.422178 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 623/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 10:02:53.427305 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131ef4f-5f3a-43f9-a770-bd90532db1e5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:53.538954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131ef4f-5f3a-43f9-a770-bd90532db1e5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:53.579881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909e19c-6016-403e-b024-e1852cf63a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:53.580597 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909e19c-6016-403e-b024-e1852cf63a9b None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:53.580948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909e19c-6016-403e-b024-e1852cf63a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:53.581223 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909e19c-6016-403e-b024-e1852cf63a9b None admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:53.582239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909e19c-6016-403e-b024-e1852cf63a9b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:53.582545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909e19c-6016-403e-b024-e1852cf63a9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:53.584208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909e19c-6016-403e-b024-e1852cf63a9b None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:53.593014 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:53.601681 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 645/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jul 5 10:02:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:53.607575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dde216-6632-414d-95cd-323c3aa0df7f None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:53.712324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dde216-6632-414d-95cd-323c3aa0df7f None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:53.752242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bc039-c649-4b4a-b7b9-5bff35d31d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:53.753092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc039-c649-4b4a-b7b9-5bff35d31d37 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:53.753357 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc039-c649-4b4a-b7b9-5bff35d31d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:53.753639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc039-c649-4b4a-b7b9-5bff35d31d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:53.766242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bc039-c649-4b4a-b7b9-5bff35d31d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:53.767708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bc039-c649-4b4a-b7b9-5bff35d31d37 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-ff7bc039-c649-4b4a-b7b9-5bff35d31d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:53.785197 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:53.786294 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:53.787411 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:53.787411 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:53.787411 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:53.787411 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:53.787411 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 624/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:53.793019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909e19c-6016-403e-b024-e1852cf63a9b None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:53.901919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909e19c-6016-403e-b024-e1852cf63a9b None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:53.936597 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb8ac3-cfc7-4e04-a05e-b6fa32ae9240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:53.937251 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb8ac3-cfc7-4e04-a05e-b6fa32ae9240 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:53.937528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb8ac3-cfc7-4e04-a05e-b6fa32ae9240 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:53.937809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb8ac3-cfc7-4e04-a05e-b6fa32ae9240 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:53.938563 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb8ac3-cfc7-4e04-a05e-b6fa32ae9240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:53.938870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb8ac3-cfc7-4e04-a05e-b6fa32ae9240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:53.940245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb8ac3-cfc7-4e04-a05e-b6fa32ae9240 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:53.951981 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 646/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:53.957998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bc039-c649-4b4a-b7b9-5bff35d31d37 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:53.995548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:54.075849 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bc039-c649-4b4a-b7b9-5bff35d31d37 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:54.115281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b8db4-a35c-4ddc-8889-e6369ebded81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:54.116088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b8db4-a35c-4ddc-8889-e6369ebded81 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:54.116374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b8db4-a35c-4ddc-8889-e6369ebded81 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:54.116658 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b8db4-a35c-4ddc-8889-e6369ebded81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:54.123851 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b8db4-a35c-4ddc-8889-e6369ebded81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:54.125328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b8db4-a35c-4ddc-8889-e6369ebded81 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-fb6b8db4-a35c-4ddc-8889-e6369ebded81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:02:54.140221 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:54.141211 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:54.142119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:02:54.142119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:02:54.142119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:02:54.142119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:02:54.142119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:02:54.142119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:02:54.142119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:54.142119 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:54.142119 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 625/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:02:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:54.149897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb8ac3-cfc7-4e04-a05e-b6fa32ae9240 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:54.216275 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:54.262681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb8ac3-cfc7-4e04-a05e-b6fa32ae9240 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:54.299640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcc494-d3d5-477a-8c33-34374f371344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:54.300304 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcc494-d3d5-477a-8c33-34374f371344 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:54.300586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcc494-d3d5-477a-8c33-34374f371344 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:54.300855 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcc494-d3d5-477a-8c33-34374f371344 None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:54.301641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcc494-d3d5-477a-8c33-34374f371344 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:54.301945 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcc494-d3d5-477a-8c33-34374f371344 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:54.303417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcc494-d3d5-477a-8c33-34374f371344 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:54.316028 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 647/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:02:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:54.322078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b8db4-a35c-4ddc-8889-e6369ebded81 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:54.420395 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:54.432023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b8db4-a35c-4ddc-8889-e6369ebded81 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:54.467723 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5dd2c-2649-4c12-bfd2-c8e5e918d6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:54.468605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5dd2c-2649-4c12-bfd2-c8e5e918d6fc None admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:54.468888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5dd2c-2649-4c12-bfd2-c8e5e918d6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:54.469163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5dd2c-2649-4c12-bfd2-c8e5e918d6fc None admin] PATH_INFO: `/v3/projects/35af178e36e243588f642edb57180d9a/users/e9d4a50f842a48b1875900426c6a1ac5/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:54.471389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5dd2c-2649-4c12-bfd2-c8e5e918d6fc None admin] RBAC: Authorizing `identity:create_grant(project_id=35af178e36e243588f642edb57180d9a, user_id=e9d4a50f842a48b1875900426c6a1ac5, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:54.472840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5dd2c-2649-4c12-bfd2-c8e5e918d6fc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:54.492680 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 626/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jul 5 10:02:54 2022] PUT /identity/v3/projects/35af178e36e243588f642edb57180d9a/users/e9d4a50f842a48b1875900426c6a1ac5/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:02:56.501008 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2170c1-d4f8-4d40-8735-cdbd103e2236 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:56.501883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2170c1-d4f8-4d40-8735-cdbd103e2236 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:56.501883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2170c1-d4f8-4d40-8735-cdbd103e2236 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:56.504144 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 648/1274] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:02:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:02:56.510466 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc740e-eb08-4f50-9cfc-256480569ab5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:56.511138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc740e-eb08-4f50-9cfc-256480569ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:56.511230 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc740e-eb08-4f50-9cfc-256480569ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:56.537104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-20fc740e-eb08-4f50-9cfc-256480569ab5 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:02:56.538846 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-20fc740e-eb08-4f50-9cfc-256480569ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:02:56.618203 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 627/1275] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:02:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:02:56.642500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2170c1-d4f8-4d40-8735-cdbd103e2236 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:56.747132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2170c1-d4f8-4d40-8735-cdbd103e2236 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:56.771467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:56.790488 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257e2bc-570e-4195-a039-0aa253d71717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:56.791305 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257e2bc-570e-4195-a039-0aa253d71717 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:56.791588 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257e2bc-570e-4195-a039-0aa253d71717 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:56.791864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257e2bc-570e-4195-a039-0aa253d71717 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:56.802418 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257e2bc-570e-4195-a039-0aa253d71717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-8257e2bc-570e-4195-a039-0aa253d71717 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 10:02:56.810102 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:56.811204 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:56.812393 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 649/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jul 5 10:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:56.820002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc740e-eb08-4f50-9cfc-256480569ab5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:56.910572 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:56.930436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc740e-eb08-4f50-9cfc-256480569ab5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:56.964262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b43d23-0500-4b97-9c25-71f86de61bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:56.964931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b43d23-0500-4b97-9c25-71f86de61bcc None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:56.965213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b43d23-0500-4b97-9c25-71f86de61bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:56.965488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b43d23-0500-4b97-9c25-71f86de61bcc None admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:56.966271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b43d23-0500-4b97-9c25-71f86de61bcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:56.966575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b43d23-0500-4b97-9c25-71f86de61bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:56.967983 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b43d23-0500-4b97-9c25-71f86de61bcc None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:56.976115 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 628/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jul 5 10:02:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:56.982877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257e2bc-570e-4195-a039-0aa253d71717 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:57.091217 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257e2bc-570e-4195-a039-0aa253d71717 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:57.126978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db948735-4eec-4437-bad9-ff65a8c85847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:57.127763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db948735-4eec-4437-bad9-ff65a8c85847 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:57.128045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db948735-4eec-4437-bad9-ff65a8c85847 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:57.128313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db948735-4eec-4437-bad9-ff65a8c85847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:57.139474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db948735-4eec-4437-bad9-ff65a8c85847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:57.140957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db948735-4eec-4437-bad9-ff65a8c85847 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-db948735-4eec-4437-bad9-ff65a8c85847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:57.158150 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:57.159113 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:57.160332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:57.160332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:57.160332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:57.160332 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:57.160332 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 650/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:57.166592 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b43d23-0500-4b97-9c25-71f86de61bcc None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:57.267893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b43d23-0500-4b97-9c25-71f86de61bcc None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:57.301512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f43b6-4ee1-430b-a82a-bbfa02b82868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:57.302157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f43b6-4ee1-430b-a82a-bbfa02b82868 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:57.302465 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f43b6-4ee1-430b-a82a-bbfa02b82868 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:57.302735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f43b6-4ee1-430b-a82a-bbfa02b82868 None admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:57.303448 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f43b6-4ee1-430b-a82a-bbfa02b82868 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:57.303760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f43b6-4ee1-430b-a82a-bbfa02b82868 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:57.305142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f43b6-4ee1-430b-a82a-bbfa02b82868 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:57.318024 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 629/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:57.326940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db948735-4eec-4437-bad9-ff65a8c85847 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:57.367819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:57.450609 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db948735-4eec-4437-bad9-ff65a8c85847 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:57.485655 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163976-3efe-4900-971d-bec1c4ae25f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:57.486493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163976-3efe-4900-971d-bec1c4ae25f5 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:57.486783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163976-3efe-4900-971d-bec1c4ae25f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:57.487055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163976-3efe-4900-971d-bec1c4ae25f5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:57.496374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2163976-3efe-4900-971d-bec1c4ae25f5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:57.497942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2163976-3efe-4900-971d-bec1c4ae25f5 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-b2163976-3efe-4900-971d-bec1c4ae25f5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 10:02:57.514385 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:57.515879 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:57.517154 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:57.517154 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:57.517154 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 10:02:57.517154 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 10:02:57.517154 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 10:02:57.517154 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 10:02:57.517154 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 10:02:57.517154 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 10:02:57.517154 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 10:02:57.517154 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:02:57.517154 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 651/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:57.522177 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f43b6-4ee1-430b-a82a-bbfa02b82868 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:57.526335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:57.631458 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f43b6-4ee1-430b-a82a-bbfa02b82868 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:57.669768 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d551bc-7ad9-42a5-b96d-bfff869d981d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:57.670465 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d551bc-7ad9-42a5-b96d-bfff869d981d None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:57.670745 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d551bc-7ad9-42a5-b96d-bfff869d981d None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:57.671015 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d551bc-7ad9-42a5-b96d-bfff869d981d None admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:57.671852 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d551bc-7ad9-42a5-b96d-bfff869d981d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:57.672161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d551bc-7ad9-42a5-b96d-bfff869d981d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:57.673597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d551bc-7ad9-42a5-b96d-bfff869d981d None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:57.691420 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 630/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jul 5 10:02:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:57.700535 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163976-3efe-4900-971d-bec1c4ae25f5 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:57.798198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:57.809116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163976-3efe-4900-971d-bec1c4ae25f5 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:57.848282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc3c50-284b-487c-a565-f31a5c11a9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:57.848946 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc3c50-284b-487c-a565-f31a5c11a9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:57.849229 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc3c50-284b-487c-a565-f31a5c11a9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:57.849499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc3c50-284b-487c-a565-f31a5c11a9e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:57.918981 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc3c50-284b-487c-a565-f31a5c11a9e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:57.920376 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc3c50-284b-487c-a565-f31a5c11a9e8 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:58.021402 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:58.032694 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 652/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jul 5 10:02:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 10:02:58.038926 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d551bc-7ad9-42a5-b96d-bfff869d981d None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:58.102772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:58.142245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d551bc-7ad9-42a5-b96d-bfff869d981d None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:58.176910 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109c91c-6bb9-49d6-afb3-41630527a584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:58.177644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109c91c-6bb9-49d6-afb3-41630527a584 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:58.177928 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109c91c-6bb9-49d6-afb3-41630527a584 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:58.178208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109c91c-6bb9-49d6-afb3-41630527a584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:58.188958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109c91c-6bb9-49d6-afb3-41630527a584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:58.190509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109c91c-6bb9-49d6-afb3-41630527a584 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-8109c91c-6bb9-49d6-afb3-41630527a584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 10:02:58.208665 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:58.209759 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:58.210818 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 10:02:58.210818 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 10:02:58.210818 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 10:02:58.210818 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:58.210818 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 631/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:58.215702 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc3c50-284b-487c-a565-f31a5c11a9e8 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:58.329487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc3c50-284b-487c-a565-f31a5c11a9e8 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:58.367617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babce112-d35d-4e69-b608-9e864989aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:58.368561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babce112-d35d-4e69-b608-9e864989aa07 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:58.369000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babce112-d35d-4e69-b608-9e864989aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:58.369424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babce112-d35d-4e69-b608-9e864989aa07 None admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:58.370298 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babce112-d35d-4e69-b608-9e864989aa07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:58.370748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babce112-d35d-4e69-b608-9e864989aa07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:58.372249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babce112-d35d-4e69-b608-9e864989aa07 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:58.384158 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 653/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jul 5 10:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:58.389833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109c91c-6bb9-49d6-afb3-41630527a584 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:58.489386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109c91c-6bb9-49d6-afb3-41630527a584 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:58.503975 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:58.524027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577f9bf-aaa2-4df1-9f56-12bf667bfdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:58.524831 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f9bf-aaa2-4df1-9f56-12bf667bfdcb None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:58.525118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f9bf-aaa2-4df1-9f56-12bf667bfdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:58.525395 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f9bf-aaa2-4df1-9f56-12bf667bfdcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:58.532715 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577f9bf-aaa2-4df1-9f56-12bf667bfdcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:58.534461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577f9bf-aaa2-4df1-9f56-12bf667bfdcb None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application [None req-e577f9bf-aaa2-4df1-9f56-12bf667bfdcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 10:02:58.550391 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application with Lock( Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:02:58.551342 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:02:58.552292 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 10:02:58.552292 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 10:02:58.552292 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 10:02:58.552292 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 10:02:58.552292 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 10:02:58.552292 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 10:02:58.552292 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 10:02:58.552292 dv-cinder-master-3-smb devstack@keystone.service[80377]: ERROR keystone.server.flask.application Jul 05 10:02:58.558326 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 632/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jul 5 10:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:02:58.558484 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babce112-d35d-4e69-b608-9e864989aa07 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:58.611443 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:02:58.660753 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babce112-d35d-4e69-b608-9e864989aa07 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:58.693262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42580af7-a8a3-4083-b2ef-212f2e7b69ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:58.693918 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42580af7-a8a3-4083-b2ef-212f2e7b69ec None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:58.694198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42580af7-a8a3-4083-b2ef-212f2e7b69ec None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:58.694499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42580af7-a8a3-4083-b2ef-212f2e7b69ec None admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:58.695284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42580af7-a8a3-4083-b2ef-212f2e7b69ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:58.695648 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42580af7-a8a3-4083-b2ef-212f2e7b69ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:58.697098 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42580af7-a8a3-4083-b2ef-212f2e7b69ec None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:58.709375 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 654/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jul 5 10:02:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:02:58.716534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577f9bf-aaa2-4df1-9f56-12bf667bfdcb None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:02:58.831200 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577f9bf-aaa2-4df1-9f56-12bf667bfdcb None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:02:58.866667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b144f-5516-444c-8f33-6bd0cef19b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:02:58.867318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b144f-5516-444c-8f33-6bd0cef19b13 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:02:58.867646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b144f-5516-444c-8f33-6bd0cef19b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:02:58.867928 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b144f-5516-444c-8f33-6bd0cef19b13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:02:58.868766 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b144f-5516-444c-8f33-6bd0cef19b13 None admin] RBAC: Adding query filter params (role.id=c184a177010144429f495f3290d28221, scope.project.id=35af178e36e243588f642edb57180d9a, user.id=e9d4a50f842a48b1875900426c6a1ac5) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:02:58.869086 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b144f-5516-444c-8f33-6bd0cef19b13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:02:58.870546 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b144f-5516-444c-8f33-6bd0cef19b13 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:02:58.876417 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 633/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jul 5 10:02:58 2022] GET /identity/v3/role_assignments?user.id=e9d4a50f842a48b1875900426c6a1ac5&scope.project.id=35af178e36e243588f642edb57180d9a&role.id=c184a177010144429f495f3290d28221 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:03:00.912107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c0e29-ede6-4fe3-842d-3f89e66057e3 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:00.912701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c0e29-ede6-4fe3-842d-3f89e66057e3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:00.912701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c0e29-ede6-4fe3-842d-3f89e66057e3 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:00.914069 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 655/1288] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:03:00.920395 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45f404-846d-494b-ad3d-d1ff8b7f82c2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:00.921107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45f404-846d-494b-ad3d-d1ff8b7f82c2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:00.921416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45f404-846d-494b-ad3d-d1ff8b7f82c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:00.942693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2b45f404-846d-494b-ad3d-d1ff8b7f82c2 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:03:00.943884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2b45f404-846d-494b-ad3d-d1ff8b7f82c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:03:00.976211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:01.014944 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 634/1289] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:03:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:03:01.037284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c0e29-ede6-4fe3-842d-3f89e66057e3 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:03:01.143194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c0e29-ede6-4fe3-842d-3f89e66057e3 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:03:01.153601 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:01.179246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd8293-2a3b-43eb-8519-cc66675a5a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:03:01.180174 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8293-2a3b-43eb-8519-cc66675a5a70 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:01.180457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8293-2a3b-43eb-8519-cc66675a5a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:01.180726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8293-2a3b-43eb-8519-cc66675a5a70 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:01.181390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd8293-2a3b-43eb-8519-cc66675a5a70 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:03:01.182819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd8293-2a3b-43eb-8519-cc66675a5a70 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-b2fd8293-2a3b-43eb-8519-cc66675a5a70 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:03:01.194828 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:03:01.196062 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:03:01.197413 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 656/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jul 5 10:03:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:03:01.203244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45f404-846d-494b-ad3d-d1ff8b7f82c2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:03:01.308655 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45f404-846d-494b-ad3d-d1ff8b7f82c2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:03:01.346142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07628fb-fe0f-40cb-a0ca-fef70f97d158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:03:01.346827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07628fb-fe0f-40cb-a0ca-fef70f97d158 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:01.347106 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07628fb-fe0f-40cb-a0ca-fef70f97d158 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:01.347374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07628fb-fe0f-40cb-a0ca-fef70f97d158 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:01.348082 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07628fb-fe0f-40cb-a0ca-fef70f97d158 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:03:01.348381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07628fb-fe0f-40cb-a0ca-fef70f97d158 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:03:01.349743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07628fb-fe0f-40cb-a0ca-fef70f97d158 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:03:01.354993 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 635/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Jul 5 10:03:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:03:01.364192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd8293-2a3b-43eb-8519-cc66675a5a70 None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:03:01.477585 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd8293-2a3b-43eb-8519-cc66675a5a70 None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:03:01.511638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981525c-f1b0-488f-a647-1a881367e46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:03:01.512301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981525c-f1b0-488f-a647-1a881367e46a None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:01.512580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981525c-f1b0-488f-a647-1a881367e46a None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:01.512838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981525c-f1b0-488f-a647-1a881367e46a None admin] PATH_INFO: `/v3/services` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:01.513542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981525c-f1b0-488f-a647-1a881367e46a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:03:01.513847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981525c-f1b0-488f-a647-1a881367e46a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:03:01.515271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981525c-f1b0-488f-a647-1a881367e46a None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:03:01.519933 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 657/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Jul 5 10:03:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:03:03.517869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab916b10-d0ee-4587-8cfc-99ddeb0db467 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:03.518503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab916b10-d0ee-4587-8cfc-99ddeb0db467 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:03.518503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab916b10-d0ee-4587-8cfc-99ddeb0db467 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:03.519810 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 636/1293] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:03:03.526130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071ee56-6ec9-4648-b9de-ab6199688b1a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:03.526949 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071ee56-6ec9-4648-b9de-ab6199688b1a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:03.527126 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071ee56-6ec9-4648-b9de-ab6199688b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:03.552107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-6071ee56-6ec9-4648-b9de-ab6199688b1a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:03:03.553332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6071ee56-6ec9-4648-b9de-ab6199688b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:03:03.585165 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:03.637034 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 658/1294] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:03:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:03:03.662660 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab916b10-d0ee-4587-8cfc-99ddeb0db467 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:03:03.700071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:03.775198 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab916b10-d0ee-4587-8cfc-99ddeb0db467 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:03:03.812324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f311d96-1723-4510-811d-344f63eed952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:03:03.812987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311d96-1723-4510-811d-344f63eed952 None admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:03.813268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311d96-1723-4510-811d-344f63eed952 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:03.813540 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311d96-1723-4510-811d-344f63eed952 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:03.814296 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f311d96-1723-4510-811d-344f63eed952 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:03:03.815682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f311d96-1723-4510-811d-344f63eed952 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:03:03.831836 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 637/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jul 5 10:03:03 2022] POST /identity/v3/services => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:03:05.834725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cf808-d287-44b6-92d2-d7964bb78cc9 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:05.835721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cf808-d287-44b6-92d2-d7964bb78cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:05.835871 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cf808-d287-44b6-92d2-d7964bb78cc9 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:05.838054 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 659/1296] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:03:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:03:05.844281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c9527-c4f2-4220-8a6c-7b2ee92058b7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:05.845057 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c9527-c4f2-4220-8a6c-7b2ee92058b7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:05.845222 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c9527-c4f2-4220-8a6c-7b2ee92058b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:05.871183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-984c9527-c4f2-4220-8a6c-7b2ee92058b7 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:03:05.872956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-984c9527-c4f2-4220-8a6c-7b2ee92058b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:03:05.953780 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 638/1297] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:03:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:03:05.982506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cf808-d287-44b6-92d2-d7964bb78cc9 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:03:06.093003 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cf808-d287-44b6-92d2-d7964bb78cc9 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:03:06.127371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1598f-d137-481a-9c20-1b8237a4061d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:03:06.128194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1598f-d137-481a-9c20-1b8237a4061d None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:06.128476 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1598f-d137-481a-9c20-1b8237a4061d None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:06.128749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1598f-d137-481a-9c20-1b8237a4061d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:06.129398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1598f-d137-481a-9c20-1b8237a4061d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:03:06.130820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1598f-d137-481a-9c20-1b8237a4061d None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-c4c1598f-d137-481a-9c20-1b8237a4061d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:03:06.142770 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:03:06.143978 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:03:06.144921 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 660/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jul 5 10:03:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:03:06.150919 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c9527-c4f2-4220-8a6c-7b2ee92058b7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:03:06.175020 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:06.265295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c9527-c4f2-4220-8a6c-7b2ee92058b7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:03:06.300548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cc40a-bfc9-4d16-ae51-3292378247b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:03:06.301218 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cc40a-bfc9-4d16-ae51-3292378247b6 None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:06.301501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cc40a-bfc9-4d16-ae51-3292378247b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:06.301775 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cc40a-bfc9-4d16-ae51-3292378247b6 None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:06.302514 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cc40a-bfc9-4d16-ae51-3292378247b6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:03:06.302820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cc40a-bfc9-4d16-ae51-3292378247b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:03:06.304231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cc40a-bfc9-4d16-ae51-3292378247b6 None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:03:06.309516 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 639/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Jul 5 10:03:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:03:06.316518 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1598f-d137-481a-9c20-1b8237a4061d None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:03:06.351112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:06.427123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1598f-d137-481a-9c20-1b8237a4061d None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:03:06.465162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9a2bd-86fc-4bf0-98bd-7f17cdfaa445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:03:06.465832 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9a2bd-86fc-4bf0-98bd-7f17cdfaa445 None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:06.466112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9a2bd-86fc-4bf0-98bd-7f17cdfaa445 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:06.466420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9a2bd-86fc-4bf0-98bd-7f17cdfaa445 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:06.467241 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9a2bd-86fc-4bf0-98bd-7f17cdfaa445 None admin] RBAC: Adding query filter params (interface=public, service_id=445f5de3d1574094b7f16533e5b0eb89, region_id=RegionOne) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:03:06.467552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9a2bd-86fc-4bf0-98bd-7f17cdfaa445 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:03:06.468953 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9a2bd-86fc-4bf0-98bd-7f17cdfaa445 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:03:06.474156 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 661/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jul 5 10:03:06 2022] GET /identity/v3/endpoints?service_id=445f5de3d1574094b7f16533e5b0eb89&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:03:06.483176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cc40a-bfc9-4d16-ae51-3292378247b6 None admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:03:06.584371 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:06.607195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cc40a-bfc9-4d16-ae51-3292378247b6 None admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:03:06.642985 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b65c2d-7dae-49dc-b87e-fb52ab36278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:03:06.643663 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b65c2d-7dae-49dc-b87e-fb52ab36278c None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:06.643940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b65c2d-7dae-49dc-b87e-fb52ab36278c None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:06.644209 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b65c2d-7dae-49dc-b87e-fb52ab36278c None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:06.644888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b65c2d-7dae-49dc-b87e-fb52ab36278c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:03:06.646258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b65c2d-7dae-49dc-b87e-fb52ab36278c None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:03:06.653093 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 640/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jul 5 10:03:06 2022] GET /identity/v3/services => generated 2191 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:03:08.671683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27037e17-6ab3-4f79-a134-cc8007bdd85b None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:08.672282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27037e17-6ab3-4f79-a134-cc8007bdd85b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:08.672282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27037e17-6ab3-4f79-a134-cc8007bdd85b None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:08.673623 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 662/1302] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:03:08.680363 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48863194-0142-458b-a872-cf41566e44d0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:08.681047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48863194-0142-458b-a872-cf41566e44d0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:08.681185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48863194-0142-458b-a872-cf41566e44d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:08.710427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-48863194-0142-458b-a872-cf41566e44d0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:03:08.711615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-48863194-0142-458b-a872-cf41566e44d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:03:08.790242 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 641/1303] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:03:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 10:03:08.819133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27037e17-6ab3-4f79-a134-cc8007bdd85b None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:03:08.920813 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:08.930967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27037e17-6ab3-4f79-a134-cc8007bdd85b None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:03:08.968398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af825ad-cf45-4e13-a8e5-614e80677b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:03:08.969627 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af825ad-cf45-4e13-a8e5-614e80677b8f None admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:08.970047 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af825ad-cf45-4e13-a8e5-614e80677b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:08.970502 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af825ad-cf45-4e13-a8e5-614e80677b8f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:08.971525 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af825ad-cf45-4e13-a8e5-614e80677b8f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:03:08.973743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af825ad-cf45-4e13-a8e5-614e80677b8f None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application [None req-8af825ad-cf45-4e13-a8e5-614e80677b8f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 10:03:08.992800 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 10:03:08.993952 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application with Lock( Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self._enter() Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application return self.creator() Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application created_value = creator( Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: ERROR keystone.server.flask.application Jul 05 10:03:08.995092 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 663/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jul 5 10:03:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 10:03:09.001980 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48863194-0142-458b-a872-cf41566e44d0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:03:09.086059 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:09.111427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48863194-0142-458b-a872-cf41566e44d0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:03:09.146240 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76b454-3369-4fe7-a14c-3ca1661483ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:03:09.146917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b454-3369-4fe7-a14c-3ca1661483ea None admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:09.147202 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b454-3369-4fe7-a14c-3ca1661483ea None admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:09.147477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b454-3369-4fe7-a14c-3ca1661483ea None admin] PATH_INFO: `/v3/services` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:09.148203 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76b454-3369-4fe7-a14c-3ca1661483ea None admin] RBAC: Adding query filter params (name=placement) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:03:09.148513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76b454-3369-4fe7-a14c-3ca1661483ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:03:09.149871 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76b454-3369-4fe7-a14c-3ca1661483ea None admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:03:09.155146 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 642/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Jul 5 10:03:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:03:09.161443 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af825ad-cf45-4e13-a8e5-614e80677b8f None admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:03:09.268245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af825ad-cf45-4e13-a8e5-614e80677b8f None admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:03:09.303519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbaca8-0575-473d-997b-20b0b8871175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:03:09.304198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbaca8-0575-473d-997b-20b0b8871175 None admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:03:09.304486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbaca8-0575-473d-997b-20b0b8871175 None admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:03:09.304760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbaca8-0575-473d-997b-20b0b8871175 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:03:09.305503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbaca8-0575-473d-997b-20b0b8871175 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:03:09.306915 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbaca8-0575-473d-997b-20b0b8871175 None admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:03:09.328861 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 664/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jul 5 10:03:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:03:19.152094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=30 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:19.324410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:11.768437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5672c-8be7-41bc-937a-1559ca2b63e8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:11.768991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5672c-8be7-41bc-937a-1559ca2b63e8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:11.768991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5672c-8be7-41bc-937a-1559ca2b63e8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:11.776086 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f7caf-d7ad-40f9-a25f-581febb2ebdd None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:11.776575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f7caf-d7ad-40f9-a25f-581febb2ebdd None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:11.776658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f7caf-d7ad-40f9-a25f-581febb2ebdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:11.780174 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 643/1307] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:11.860205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-773f7caf-d7ad-40f9-a25f-581febb2ebdd None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:11.861437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-773f7caf-d7ad-40f9-a25f-581febb2ebdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:12.027019 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 665/1308] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:12.081185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a0ae0-627b-441f-b589-0e04ade9cdc4 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:12.081751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a0ae0-627b-441f-b589-0e04ade9cdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:12.082263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a0ae0-627b-441f-b589-0e04ade9cdc4 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:12.084303 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 644/1309] 10.222.0.16 () {54 vars in 964 bytes} [Tue Jul 5 10:04:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:12.090816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf3fc0-a222-4a05-9a3a-a1d0a2dbc48d None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:12.091384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf3fc0-a222-4a05-9a3a-a1d0a2dbc48d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:12.091821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf3fc0-a222-4a05-9a3a-a1d0a2dbc48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:12.352059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3ddf3fc0-a222-4a05-9a3a-a1d0a2dbc48d None None] MFA Rules not processed for user `e0865f21da0a4c95a84c0099a1b9a78b`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:12.353737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3ddf3fc0-a222-4a05-9a3a-a1d0a2dbc48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:12.384238 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:12.452360 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 666/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Tue Jul 5 10:04:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:12.461401 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a0ae0-627b-441f-b589-0e04ade9cdc4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:12.463197 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-160a0ae0-627b-441f-b589-0e04ade9cdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:12.583416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a0ae0-627b-441f-b589-0e04ade9cdc4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:12.660259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6a0b1-8960-4544-afcf-f55c65f4355b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:12.660934 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a0b1-8960-4544-afcf-f55c65f4355b service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:12.661218 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a0b1-8960-4544-afcf-f55c65f4355b service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:12.661496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a0b1-8960-4544-afcf-f55c65f4355b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:12.663339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3dd6a0b1-8960-4544-afcf-f55c65f4355b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:12.724502 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:12.787224 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6a0b1-8960-4544-afcf-f55c65f4355b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:12.788845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6a0b1-8960-4544-afcf-f55c65f4355b service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:12.814690 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 645/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:15.155583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96de5d3-9312-4ebd-a144-32dfb0fe126b None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:15.156123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96de5d3-9312-4ebd-a144-32dfb0fe126b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:15.156123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96de5d3-9312-4ebd-a144-32dfb0fe126b None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:15.157403 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 667/1312] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:15.162889 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23955b6d-f950-48bc-a3f8-a97a20df29cf None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:15.163346 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23955b6d-f950-48bc-a3f8-a97a20df29cf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:15.163443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23955b6d-f950-48bc-a3f8-a97a20df29cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:15.234151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-23955b6d-f950-48bc-a3f8-a97a20df29cf None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:15.235398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-23955b6d-f950-48bc-a3f8-a97a20df29cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:15.336689 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 646/1313] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:15.387243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6836a-fa47-4f4d-9177-9229e4f7f05e None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:15.387567 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6836a-fa47-4f4d-9177-9229e4f7f05e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:15.387840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6836a-fa47-4f4d-9177-9229e4f7f05e None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:15.389255 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 668/1314] 10.222.0.16 () {54 vars in 964 bytes} [Tue Jul 5 10:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:15.395776 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac15437-d0ad-42f5-8c36-7e666d1eb4f2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:15.396043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac15437-d0ad-42f5-8c36-7e666d1eb4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:15.396346 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac15437-d0ad-42f5-8c36-7e666d1eb4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:15.407933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:15.481752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dac15437-d0ad-42f5-8c36-7e666d1eb4f2 None None] MFA Rules not processed for user `e0865f21da0a4c95a84c0099a1b9a78b`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:15.483062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dac15437-d0ad-42f5-8c36-7e666d1eb4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:15.574325 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 647/1315] 10.222.0.16 () {58 vars in 1061 bytes} [Tue Jul 5 10:04:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:15.583367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6836a-fa47-4f4d-9177-9229e4f7f05e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:15.585098 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b1e6836a-fa47-4f4d-9177-9229e4f7f05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:15.692457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6836a-fa47-4f4d-9177-9229e4f7f05e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:15.773330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:15.776565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b3099-e9f5-42f1-acb3-818e78790dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:15.777492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b3099-e9f5-42f1-acb3-818e78790dfb service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:15.777840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b3099-e9f5-42f1-acb3-818e78790dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:15.778084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b3099-e9f5-42f1-acb3-818e78790dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:15.780377 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-907b3099-e9f5-42f1-acb3-818e78790dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:15.889104 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b3099-e9f5-42f1-acb3-818e78790dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:15.890606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b3099-e9f5-42f1-acb3-818e78790dfb service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:15.909314 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 669/1316] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:18.427468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051be43-6b14-469c-9580-f1673afb22eb None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:18.428068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051be43-6b14-469c-9580-f1673afb22eb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:18.428068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051be43-6b14-469c-9580-f1673afb22eb None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:18.429455 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 648/1317] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:18.437970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef8e26-eaeb-4da9-8749-c5cc705e76dd None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:18.438433 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef8e26-eaeb-4da9-8749-c5cc705e76dd None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:18.438572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef8e26-eaeb-4da9-8749-c5cc705e76dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:18.511736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b1ef8e26-eaeb-4da9-8749-c5cc705e76dd None None] MFA Rules not processed for user `e94fb5eec4f149e686cb01446e186584`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:18.512958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b1ef8e26-eaeb-4da9-8749-c5cc705e76dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:18.565736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:18.617491 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 670/1318] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:18.661312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051be43-6b14-469c-9580-f1673afb22eb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:18.662997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6051be43-6b14-469c-9580-f1673afb22eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:18.777237 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051be43-6b14-469c-9580-f1673afb22eb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:18.780181 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:18.853837 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f9406-9330-4e6e-b977-3c2befe5eae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:18.854545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f9406-9330-4e6e-b977-3c2befe5eae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:18.854839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f9406-9330-4e6e-b977-3c2befe5eae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:18.855106 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f9406-9330-4e6e-b977-3c2befe5eae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:18.856776 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7c7f9406-9330-4e6e-b977-3c2befe5eae5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:18.968668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f9406-9330-4e6e-b977-3c2befe5eae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:18.970101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f9406-9330-4e6e-b977-3c2befe5eae5 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:18.988693 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 649/1319] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:21.535196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaf7f8-06c7-4b38-a50e-69859a0fef82 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:21.535799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaf7f8-06c7-4b38-a50e-69859a0fef82 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:21.535799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaf7f8-06c7-4b38-a50e-69859a0fef82 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:21.537124 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 671/1320] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:21.545865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e6549-1159-46eb-88de-08f899477c03 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:21.546359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e6549-1159-46eb-88de-08f899477c03 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:21.546474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e6549-1159-46eb-88de-08f899477c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:21.587139 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:21.612434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a89e6549-1159-46eb-88de-08f899477c03 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:21.613666 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a89e6549-1159-46eb-88de-08f899477c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:21.710824 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 650/1321] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:04:21.748834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaf7f8-06c7-4b38-a50e-69859a0fef82 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:21.750404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7ffaf7f8-06c7-4b38-a50e-69859a0fef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:21.868890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaf7f8-06c7-4b38-a50e-69859a0fef82 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:21.953778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f31cb-4586-4af9-a0bf-7727c4ae778a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:21.954460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f31cb-4586-4af9-a0bf-7727c4ae778a service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:21.954740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f31cb-4586-4af9-a0bf-7727c4ae778a service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:21.955021 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f31cb-4586-4af9-a0bf-7727c4ae778a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:21.956684 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b45f31cb-4586-4af9-a0bf-7727c4ae778a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:21.964273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:22.073196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f31cb-4586-4af9-a0bf-7727c4ae778a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:22.074681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f31cb-4586-4af9-a0bf-7727c4ae778a service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:22.094170 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 672/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:04:24.816041 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4528f2b-ecee-45ec-8140-6f3e3b04c4a8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:24.816600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4528f2b-ecee-45ec-8140-6f3e3b04c4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:24.816600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4528f2b-ecee-45ec-8140-6f3e3b04c4a8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:24.818021 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 651/1323] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:24.823724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0386036-195a-4e34-bc14-ee1e5be10ed2 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:24.824188 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0386036-195a-4e34-bc14-ee1e5be10ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:24.824298 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0386036-195a-4e34-bc14-ee1e5be10ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:24.895350 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-d0386036-195a-4e34-bc14-ee1e5be10ed2 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:24.896714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d0386036-195a-4e34-bc14-ee1e5be10ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:24.965691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:25.012609 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 673/1324] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:25.049153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4528f2b-ecee-45ec-8140-6f3e3b04c4a8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:25.050816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e4528f2b-ecee-45ec-8140-6f3e3b04c4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:25.163823 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4528f2b-ecee-45ec-8140-6f3e3b04c4a8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:25.177207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:25.239867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a470cdb-1fec-452f-9935-dcb5544e702f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:25.240595 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470cdb-1fec-452f-9935-dcb5544e702f service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:25.240856 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470cdb-1fec-452f-9935-dcb5544e702f service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:25.241182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470cdb-1fec-452f-9935-dcb5544e702f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:25.242873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6a470cdb-1fec-452f-9935-dcb5544e702f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:25.357982 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a470cdb-1fec-452f-9935-dcb5544e702f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:25.359328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a470cdb-1fec-452f-9935-dcb5544e702f service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:25.378389 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 652/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:27.442121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1333d82-5b42-4d27-96da-2b981fdd82b2 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:27.442722 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1333d82-5b42-4d27-96da-2b981fdd82b2 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:27.442722 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1333d82-5b42-4d27-96da-2b981fdd82b2 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:27.444005 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 674/1326] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:27.449318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef6cf6-4969-4cc9-9619-ebe6ee3f4bd0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:27.449749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef6cf6-4969-4cc9-9619-ebe6ee3f4bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:27.449860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef6cf6-4969-4cc9-9619-ebe6ee3f4bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:27.496798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:27.515439 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-34ef6cf6-4969-4cc9-9619-ebe6ee3f4bd0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:27.516639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-34ef6cf6-4969-4cc9-9619-ebe6ee3f4bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:27.615943 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 653/1327] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:27.651166 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1333d82-5b42-4d27-96da-2b981fdd82b2 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:27.652712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b1333d82-5b42-4d27-96da-2b981fdd82b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:27.759258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1333d82-5b42-4d27-96da-2b981fdd82b2 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:27.841470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a06728-994e-44ff-b0ad-b3b285445f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:27.842135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a06728-994e-44ff-b0ad-b3b285445f8c service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:27.842529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a06728-994e-44ff-b0ad-b3b285445f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:27.842801 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a06728-994e-44ff-b0ad-b3b285445f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:27.844461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e7a06728-994e-44ff-b0ad-b3b285445f8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:27.862601 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:27.958348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a06728-994e-44ff-b0ad-b3b285445f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:27.959887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a06728-994e-44ff-b0ad-b3b285445f8c service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:27.978534 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 675/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:30.040749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f4797-2910-4201-bd7f-b717b79c9458 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:30.041332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f4797-2910-4201-bd7f-b717b79c9458 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:30.041332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f4797-2910-4201-bd7f-b717b79c9458 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:30.042616 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 654/1329] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:30.047823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc87e6-7e36-48eb-85f2-fdc507e5c404 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:30.048250 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc87e6-7e36-48eb-85f2-fdc507e5c404 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:30.048343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc87e6-7e36-48eb-85f2-fdc507e5c404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:30.117859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-90dc87e6-7e36-48eb-85f2-fdc507e5c404 None None] MFA Rules not processed for user `e94fb5eec4f149e686cb01446e186584`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:30.119122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-90dc87e6-7e36-48eb-85f2-fdc507e5c404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:30.186413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:30.221003 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 676/1330] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:30.258099 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f4797-2910-4201-bd7f-b717b79c9458 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:30.259668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-881f4797-2910-4201-bd7f-b717b79c9458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:30.368085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f4797-2910-4201-bd7f-b717b79c9458 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:30.390369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:30.445940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a32f25-76c6-4788-af6c-94229bfbe336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:30.446631 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a32f25-76c6-4788-af6c-94229bfbe336 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:30.446912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a32f25-76c6-4788-af6c-94229bfbe336 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:30.447183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a32f25-76c6-4788-af6c-94229bfbe336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:30.448825 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-79a32f25-76c6-4788-af6c-94229bfbe336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:30.572374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a32f25-76c6-4788-af6c-94229bfbe336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:30.573834 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a32f25-76c6-4788-af6c-94229bfbe336 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:30.595357 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 655/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:32.841346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60a4e8-752a-4928-a5ea-6ec83f573cca None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:32.841912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60a4e8-752a-4928-a5ea-6ec83f573cca None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:32.841912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60a4e8-752a-4928-a5ea-6ec83f573cca None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:32.843314 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 677/1332] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:32.848756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9165d-c7f4-4e98-a6b9-9a3b776e2165 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:32.849224 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9165d-c7f4-4e98-a6b9-9a3b776e2165 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:32.849303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9165d-c7f4-4e98-a6b9-9a3b776e2165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:32.917754 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:32.921066 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f4c9165d-c7f4-4e98-a6b9-9a3b776e2165 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:32.922885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f4c9165d-c7f4-4e98-a6b9-9a3b776e2165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:33.031823 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 656/1333] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:33.069985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60a4e8-752a-4928-a5ea-6ec83f573cca None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:33.071571 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-aa60a4e8-752a-4928-a5ea-6ec83f573cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:33.185091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60a4e8-752a-4928-a5ea-6ec83f573cca None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:33.263480 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc50c2-d628-4826-a1e4-cf764202f9fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:33.264135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc50c2-d628-4826-a1e4-cf764202f9fb service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:33.264418 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc50c2-d628-4826-a1e4-cf764202f9fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:33.264689 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc50c2-d628-4826-a1e4-cf764202f9fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:33.271580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d6dc50c2-d628-4826-a1e4-cf764202f9fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:33.302518 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:33.389177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc50c2-d628-4826-a1e4-cf764202f9fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:33.390673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc50c2-d628-4826-a1e4-cf764202f9fb service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:33.409618 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 678/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:35.462960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737cc06-dcfa-41c7-b0f2-b44bb6eb93f8 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:35.463821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737cc06-dcfa-41c7-b0f2-b44bb6eb93f8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:35.463975 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737cc06-dcfa-41c7-b0f2-b44bb6eb93f8 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:35.466064 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 657/1335] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:35.474364 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15146ac1-a26e-429e-ba38-b33adc0c574c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:35.474814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15146ac1-a26e-429e-ba38-b33adc0c574c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:35.474906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15146ac1-a26e-429e-ba38-b33adc0c574c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:35.542974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-15146ac1-a26e-429e-ba38-b33adc0c574c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:35.544202 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-15146ac1-a26e-429e-ba38-b33adc0c574c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:35.637710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:35.665407 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 679/1336] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:35.705866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737cc06-dcfa-41c7-b0f2-b44bb6eb93f8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:35.708224 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2737cc06-dcfa-41c7-b0f2-b44bb6eb93f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:35.824252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737cc06-dcfa-41c7-b0f2-b44bb6eb93f8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:35.861331 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:35.914573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ad173-7869-4c4e-a307-44cfbe8a339c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:35.915229 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad173-7869-4c4e-a307-44cfbe8a339c service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:35.915521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad173-7869-4c4e-a307-44cfbe8a339c service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:35.915793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad173-7869-4c4e-a307-44cfbe8a339c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:35.917468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ed4ad173-7869-4c4e-a307-44cfbe8a339c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:36.038030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ad173-7869-4c4e-a307-44cfbe8a339c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:36.039416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ad173-7869-4c4e-a307-44cfbe8a339c service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:36.058078 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 658/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:38.395471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0c99e-01ed-4815-9bf3-008251213c73 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:38.396085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0c99e-01ed-4815-9bf3-008251213c73 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:38.396085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0c99e-01ed-4815-9bf3-008251213c73 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:38.397416 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 680/1338] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:38.403712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1639-4f24-4dee-8729-3a381a7e3d73 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:38.404165 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1639-4f24-4dee-8729-3a381a7e3d73 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:38.404267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1639-4f24-4dee-8729-3a381a7e3d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:38.484575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-19ce1639-4f24-4dee-8729-3a381a7e3d73 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:38.485790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-19ce1639-4f24-4dee-8729-3a381a7e3d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:38.494518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:38.597028 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 659/1339] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:38.638854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0c99e-01ed-4815-9bf3-008251213c73 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:38.640397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-52f0c99e-01ed-4815-9bf3-008251213c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:38.756617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0c99e-01ed-4815-9bf3-008251213c73 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:38.840341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d36684-a37b-4619-923a-8316f3e98e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:38.841003 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d36684-a37b-4619-923a-8316f3e98e21 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:38.841290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d36684-a37b-4619-923a-8316f3e98e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:38.841564 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d36684-a37b-4619-923a-8316f3e98e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:38.843235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-90d36684-a37b-4619-923a-8316f3e98e21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:38.885181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:38.958559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d36684-a37b-4619-923a-8316f3e98e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:38.960008 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d36684-a37b-4619-923a-8316f3e98e21 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:38.978097 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 681/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:42.929544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9e7d-c3ba-4e54-ba74-c95b0543e87a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:42.930115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9e7d-c3ba-4e54-ba74-c95b0543e87a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:42.930115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9e7d-c3ba-4e54-ba74-c95b0543e87a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:42.931489 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 660/1341] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:42.937076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a35f23-0da1-4f75-bbb5-581d62032800 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:42.937486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a35f23-0da1-4f75-bbb5-581d62032800 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:42.937650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a35f23-0da1-4f75-bbb5-581d62032800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:43.015174 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-39a35f23-0da1-4f75-bbb5-581d62032800 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:43.016390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-39a35f23-0da1-4f75-bbb5-581d62032800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:43.113675 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:43.128030 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 682/1342] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:43.166539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e9e7d-c3ba-4e54-ba74-c95b0543e87a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:43.168820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0f8e9e7d-c3ba-4e54-ba74-c95b0543e87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:43.268184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e9e7d-c3ba-4e54-ba74-c95b0543e87a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:43.272598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a9858-6423-4330-947f-66d30adc9bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:43.273251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a9858-6423-4330-947f-66d30adc9bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:43.273532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a9858-6423-4330-947f-66d30adc9bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:43.273801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a9858-6423-4330-947f-66d30adc9bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:43.283234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a9858-6423-4330-947f-66d30adc9bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:43.284682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a9858-6423-4330-947f-66d30adc9bd5 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:43.294520 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 661/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:45.954332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6ba07-d3a2-4297-acce-cc0ee1df3f33 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:45.954947 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6ba07-d3a2-4297-acce-cc0ee1df3f33 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:45.954947 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6ba07-d3a2-4297-acce-cc0ee1df3f33 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:45.956325 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 683/1344] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:45.962517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0311-7e19-4f61-a2b7-f24dc83af6ee None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:45.962980 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0311-7e19-4f61-a2b7-f24dc83af6ee None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:45.963081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0311-7e19-4f61-a2b7-f24dc83af6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:45.993387 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c31b0311-7e19-4f61-a2b7-f24dc83af6ee None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:45.994632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c31b0311-7e19-4f61-a2b7-f24dc83af6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:46.023756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:46.081105 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 662/1345] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:46.119397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6ba07-d3a2-4297-acce-cc0ee1df3f33 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:46.131406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6ba07-d3a2-4297-acce-cc0ee1df3f33 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:46.135366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06929a8-2fcc-4a7f-8c5b-3e4b7ca7551c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:46.136046 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06929a8-2fcc-4a7f-8c5b-3e4b7ca7551c service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:46.136323 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06929a8-2fcc-4a7f-8c5b-3e4b7ca7551c service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:46.136596 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06929a8-2fcc-4a7f-8c5b-3e4b7ca7551c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:46.147574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06929a8-2fcc-4a7f-8c5b-3e4b7ca7551c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:46.149053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06929a8-2fcc-4a7f-8c5b-3e4b7ca7551c service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:46.160268 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 684/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:50.114824 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430dc2d-0baf-4adc-b66a-78f3b313f1a0 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:50.115383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430dc2d-0baf-4adc-b66a-78f3b313f1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:50.115383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430dc2d-0baf-4adc-b66a-78f3b313f1a0 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:50.116806 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 663/1347] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:50.122249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32ec8b-0026-4d74-a505-baf53d73930a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:50.122694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32ec8b-0026-4d74-a505-baf53d73930a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:50.122931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32ec8b-0026-4d74-a505-baf53d73930a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:50.141740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-2b32ec8b-0026-4d74-a505-baf53d73930a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:50.143078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2b32ec8b-0026-4d74-a505-baf53d73930a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:50.151293 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 685/1348] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:50.190370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430dc2d-0baf-4adc-b66a-78f3b313f1a0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:04:50.202100 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430dc2d-0baf-4adc-b66a-78f3b313f1a0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:04:50.205593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c5504-c899-431c-a658-c82b43cd5309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:04:50.206257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c5504-c899-431c-a658-c82b43cd5309 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:50.206596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c5504-c899-431c-a658-c82b43cd5309 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:50.206880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c5504-c899-431c-a658-c82b43cd5309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:50.216134 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c5504-c899-431c-a658-c82b43cd5309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:04:50.217569 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c5504-c899-431c-a658-c82b43cd5309 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:04:50.227053 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 664/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:55.201365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600bf36-9bb4-4616-849a-f206528ef64f None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:55.201917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600bf36-9bb4-4616-849a-f206528ef64f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:55.202177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600bf36-9bb4-4616-849a-f206528ef64f None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:55.204031 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 686/1350] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:04:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:55.210069 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4d297-b810-447b-a3bb-a1c07b22390a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:55.210578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4d297-b810-447b-a3bb-a1c07b22390a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:55.210879 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4d297-b810-447b-a3bb-a1c07b22390a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:55.292786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-90d4d297-b810-447b-a3bb-a1c07b22390a None None] MFA Rules not processed for user `e9d4a50f842a48b1875900426c6a1ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:55.294600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-90d4d297-b810-447b-a3bb-a1c07b22390a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:55.434860 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 665/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:04:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:04:59.680969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a07c0-0221-4463-8a1a-c6c3be896a16 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:59.681540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a07c0-0221-4463-8a1a-c6c3be896a16 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:59.681540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a07c0-0221-4463-8a1a-c6c3be896a16 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:59.683352 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 687/1352] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:04:59.688772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e98b5-48b0-48dd-96a9-fc3f0d8c10ee None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:04:59.689668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e98b5-48b0-48dd-96a9-fc3f0d8c10ee None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:04:59.690187 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e98b5-48b0-48dd-96a9-fc3f0d8c10ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:04:59.712510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-970e98b5-48b0-48dd-96a9-fc3f0d8c10ee None None] MFA Rules not processed for user `e9d4a50f842a48b1875900426c6a1ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:04:59.714344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-970e98b5-48b0-48dd-96a9-fc3f0d8c10ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:04:59.724583 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 666/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:04:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:00.136255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=13 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:05:01.052193 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555d877-43a6-4118-8e16-ff54ca775359 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:01.052879 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555d877-43a6-4118-8e16-ff54ca775359 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:01.052879 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555d877-43a6-4118-8e16-ff54ca775359 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:01.054000 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 688/1354] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:01.059091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcedf39-cb77-4c43-90e0-39f7b0c3f78b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:01.059534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcedf39-cb77-4c43-90e0-39f7b0c3f78b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:01.059680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcedf39-cb77-4c43-90e0-39f7b0c3f78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:01.079509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7bcedf39-cb77-4c43-90e0-39f7b0c3f78b None None] MFA Rules not processed for user `e9d4a50f842a48b1875900426c6a1ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:01.080741 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7bcedf39-cb77-4c43-90e0-39f7b0c3f78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:01.090910 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 667/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:05:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:04.765211 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff8d48-5bec-47ab-a8fe-e3413d1fe751 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:04.766037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff8d48-5bec-47ab-a8fe-e3413d1fe751 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:04.766505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff8d48-5bec-47ab-a8fe-e3413d1fe751 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:04.774112 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 689/1356] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:04.776468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95bb5f-9646-4e29-b40a-c8174960dda0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:04.777303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95bb5f-9646-4e29-b40a-c8174960dda0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:04.777787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95bb5f-9646-4e29-b40a-c8174960dda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:04.797329 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6c95bb5f-9646-4e29-b40a-c8174960dda0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:04.798937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6c95bb5f-9646-4e29-b40a-c8174960dda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:04.810680 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 668/1357] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:04.863515 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e37e2a-2e51-4985-bbec-470dbdb02224 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:04.864179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e37e2a-2e51-4985-bbec-470dbdb02224 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:04.864697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e37e2a-2e51-4985-bbec-470dbdb02224 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:04.866815 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 690/1358] 10.222.0.16 () {54 vars in 961 bytes} [Tue Jul 5 10:05:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:04.882683 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664425f4-5b41-427c-b200-0bfc45bdb856 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:04.882979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664425f4-5b41-427c-b200-0bfc45bdb856 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:04.883314 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664425f4-5b41-427c-b200-0bfc45bdb856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:04.932024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:05:04.940978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-664425f4-5b41-427c-b200-0bfc45bdb856 None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:04.942625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-664425f4-5b41-427c-b200-0bfc45bdb856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:05.026048 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 669/1359] 10.222.0.16 () {58 vars in 1058 bytes} [Tue Jul 5 10:05:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:05.034115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e37e2a-2e51-4985-bbec-470dbdb02224 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:05.051515 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e37e2a-2e51-4985-bbec-470dbdb02224 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:05.057175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19336cea-400a-475d-9983-29edf5ec7f93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:05.057864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19336cea-400a-475d-9983-29edf5ec7f93 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:05.058151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19336cea-400a-475d-9983-29edf5ec7f93 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:05.058460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19336cea-400a-475d-9983-29edf5ec7f93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:05.067303 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19336cea-400a-475d-9983-29edf5ec7f93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:05.068918 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19336cea-400a-475d-9983-29edf5ec7f93 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:05.078666 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 691/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:07.293702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434949d6-0018-4228-8c0d-2bdc9b70d7c2 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:07.294355 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434949d6-0018-4228-8c0d-2bdc9b70d7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:07.294461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434949d6-0018-4228-8c0d-2bdc9b70d7c2 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:07.296124 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 670/1361] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:07.301384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc771f0-6a52-4968-b685-08769129656b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:07.301859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc771f0-6a52-4968-b685-08769129656b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:07.302146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc771f0-6a52-4968-b685-08769129656b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:07.322500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ebc771f0-6a52-4968-b685-08769129656b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:07.323857 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ebc771f0-6a52-4968-b685-08769129656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:07.332869 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 692/1362] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:07.377287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434949d6-0018-4228-8c0d-2bdc9b70d7c2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:07.393201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434949d6-0018-4228-8c0d-2bdc9b70d7c2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:07.399136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42c6b1-8fe1-4edc-b774-8112af5848c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:07.399787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42c6b1-8fe1-4edc-b774-8112af5848c2 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:07.400073 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42c6b1-8fe1-4edc-b774-8112af5848c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:07.400349 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42c6b1-8fe1-4edc-b774-8112af5848c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:07.409350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42c6b1-8fe1-4edc-b774-8112af5848c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:07.411067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42c6b1-8fe1-4edc-b774-8112af5848c2 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:07.421395 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 671/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:09.749773 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6694e7-c8dc-4507-9cef-764034a51729 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:09.750546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6694e7-c8dc-4507-9cef-764034a51729 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:09.750546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6694e7-c8dc-4507-9cef-764034a51729 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:09.751752 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 693/1364] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:09.759762 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df246b3b-0207-40f4-aa7a-7ae9ec095708 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:09.760184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df246b3b-0207-40f4-aa7a-7ae9ec095708 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:09.760330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df246b3b-0207-40f4-aa7a-7ae9ec095708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:09.788662 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-df246b3b-0207-40f4-aa7a-7ae9ec095708 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:09.789906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-df246b3b-0207-40f4-aa7a-7ae9ec095708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:09.799450 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 672/1365] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:09.844256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1eb8e-3f56-4b6c-8f90-7b41784e7c97 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:09.844536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1eb8e-3f56-4b6c-8f90-7b41784e7c97 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:09.844813 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1eb8e-3f56-4b6c-8f90-7b41784e7c97 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:09.846181 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 694/1366] 10.222.0.16 () {54 vars in 961 bytes} [Tue Jul 5 10:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:09.851603 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0163a-e856-4e48-9062-e1ff812aaca3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:09.851917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0163a-e856-4e48-9062-e1ff812aaca3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:09.852192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0163a-e856-4e48-9062-e1ff812aaca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:09.871305 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-83d0163a-e856-4e48-9062-e1ff812aaca3 None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:09.872485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-83d0163a-e856-4e48-9062-e1ff812aaca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:09.882549 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 673/1367] 10.222.0.16 () {58 vars in 1058 bytes} [Tue Jul 5 10:05:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:05:09.890530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1eb8e-3f56-4b6c-8f90-7b41784e7c97 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:09.906328 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1eb8e-3f56-4b6c-8f90-7b41784e7c97 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:09.912146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576443a4-cfdf-41fb-8047-03ebc52a53aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:09.912787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576443a4-cfdf-41fb-8047-03ebc52a53aa service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:09.913065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576443a4-cfdf-41fb-8047-03ebc52a53aa service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:09.913337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576443a4-cfdf-41fb-8047-03ebc52a53aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:09.921428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576443a4-cfdf-41fb-8047-03ebc52a53aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:09.923053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576443a4-cfdf-41fb-8047-03ebc52a53aa service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:09.932354 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 695/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:12.010844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35eda31-5663-4ddf-8cc2-f5a56de8dd45 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:12.011415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35eda31-5663-4ddf-8cc2-f5a56de8dd45 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:12.011415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35eda31-5663-4ddf-8cc2-f5a56de8dd45 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:12.012993 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 674/1369] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:12.018339 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a693c-e81e-4498-96be-8bd0ef358cb5 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:12.019250 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a693c-e81e-4498-96be-8bd0ef358cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:12.019582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a693c-e81e-4498-96be-8bd0ef358cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:12.039222 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3e2a693c-e81e-4498-96be-8bd0ef358cb5 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:12.040478 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3e2a693c-e81e-4498-96be-8bd0ef358cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:12.050771 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 696/1370] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:12.097194 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35eda31-5663-4ddf-8cc2-f5a56de8dd45 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:12.115644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35eda31-5663-4ddf-8cc2-f5a56de8dd45 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:12.122107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524eb84-28d0-4492-858b-e837e81f9e83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:12.122810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524eb84-28d0-4492-858b-e837e81f9e83 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:12.123092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524eb84-28d0-4492-858b-e837e81f9e83 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:12.123364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524eb84-28d0-4492-858b-e837e81f9e83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:12.133069 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524eb84-28d0-4492-858b-e837e81f9e83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:12.134692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524eb84-28d0-4492-858b-e837e81f9e83 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:12.145874 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 675/1371] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:12.750325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2066713-e990-430a-af62-cfcabcd49a6d None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:12.750884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2066713-e990-430a-af62-cfcabcd49a6d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:12.751344 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2066713-e990-430a-af62-cfcabcd49a6d None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:12.752709 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 697/1372] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:12.759148 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460c5f-ac89-4290-b9ea-14b5ad55c896 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:12.759974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460c5f-ac89-4290-b9ea-14b5ad55c896 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:12.760425 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460c5f-ac89-4290-b9ea-14b5ad55c896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:12.781461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-11460c5f-ac89-4290-b9ea-14b5ad55c896 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:12.782703 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-11460c5f-ac89-4290-b9ea-14b5ad55c896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:12.794491 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 676/1373] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:13.130577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb499ff-8a58-446a-a03c-19ed3508f395 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:13.130577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb499ff-8a58-446a-a03c-19ed3508f395 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:13.130577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb499ff-8a58-446a-a03c-19ed3508f395 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:13.135061 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 698/1374] 10.222.0.16 () {54 vars in 963 bytes} [Tue Jul 5 10:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:13.137830 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373695a2-30e5-4a34-b514-10c8fd39a516 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:13.137830 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373695a2-30e5-4a34-b514-10c8fd39a516 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:13.137830 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373695a2-30e5-4a34-b514-10c8fd39a516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:13.188042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-373695a2-30e5-4a34-b514-10c8fd39a516 None None] MFA Rules not processed for user `dc11037d649241608ea1378cf4692dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:13.189253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-373695a2-30e5-4a34-b514-10c8fd39a516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:13.251100 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 677/1375] 10.222.0.16 () {58 vars in 1060 bytes} [Tue Jul 5 10:05:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:13.258728 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb499ff-8a58-446a-a03c-19ed3508f395 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:13.269966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb499ff-8a58-446a-a03c-19ed3508f395 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:13.277153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410e2cf-b1d7-4bda-8b3a-e550756e0378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:13.277988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410e2cf-b1d7-4bda-8b3a-e550756e0378 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:13.278296 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410e2cf-b1d7-4bda-8b3a-e550756e0378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:13.278575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410e2cf-b1d7-4bda-8b3a-e550756e0378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:13.288088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410e2cf-b1d7-4bda-8b3a-e550756e0378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:13.289559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410e2cf-b1d7-4bda-8b3a-e550756e0378 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:13.300632 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 699/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:14.230547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494ab2-ac6d-463f-bd58-a0845030b34d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:14.231101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494ab2-ac6d-463f-bd58-a0845030b34d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:14.231199 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494ab2-ac6d-463f-bd58-a0845030b34d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:14.232825 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 678/1377] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:14.238109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716d0c6-2eb3-451a-97bb-b946f5b7ac64 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:14.238661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716d0c6-2eb3-451a-97bb-b946f5b7ac64 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:14.238928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716d0c6-2eb3-451a-97bb-b946f5b7ac64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:14.258568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-4716d0c6-2eb3-451a-97bb-b946f5b7ac64 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:14.259867 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4716d0c6-2eb3-451a-97bb-b946f5b7ac64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:14.268897 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 700/1378] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:14.313748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03494ab2-ac6d-463f-bd58-a0845030b34d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:14.328551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03494ab2-ac6d-463f-bd58-a0845030b34d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:14.334362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c692e-cf6d-45e2-a925-3009ab9e571a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:14.334987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c692e-cf6d-45e2-a925-3009ab9e571a service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:14.335267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c692e-cf6d-45e2-a925-3009ab9e571a service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:14.335539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c692e-cf6d-45e2-a925-3009ab9e571a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:14.343646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c692e-cf6d-45e2-a925-3009ab9e571a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:14.345264 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c692e-cf6d-45e2-a925-3009ab9e571a service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:14.349123 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:05:14.355763 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 679/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:16.372634 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be1989-c3ed-40a6-a605-1fde3711fbd0 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:16.373192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be1989-c3ed-40a6-a605-1fde3711fbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:16.373192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be1989-c3ed-40a6-a605-1fde3711fbd0 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:16.374571 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 701/1380] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:16.380289 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d3a53-288e-4014-ae75-e6d990f7165b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:16.380762 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d3a53-288e-4014-ae75-e6d990f7165b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:16.380908 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d3a53-288e-4014-ae75-e6d990f7165b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:16.399396 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d53d3a53-288e-4014-ae75-e6d990f7165b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:16.400594 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d53d3a53-288e-4014-ae75-e6d990f7165b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:16.410322 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 680/1381] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:16.455598 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be1989-c3ed-40a6-a605-1fde3711fbd0 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:16.472959 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be1989-c3ed-40a6-a605-1fde3711fbd0 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:16.478186 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64167ad5-5363-40f6-baca-abaca756f066 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:16.478866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64167ad5-5363-40f6-baca-abaca756f066 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:16.479148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64167ad5-5363-40f6-baca-abaca756f066 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:16.479422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64167ad5-5363-40f6-baca-abaca756f066 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:16.488173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64167ad5-5363-40f6-baca-abaca756f066 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:16.489756 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64167ad5-5363-40f6-baca-abaca756f066 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:16.498857 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 702/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:18.562968 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2165d-94e6-4cef-b587-60b14db21929 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:18.563543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2165d-94e6-4cef-b587-60b14db21929 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:18.564084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2165d-94e6-4cef-b587-60b14db21929 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:18.565607 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 681/1383] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:18.571734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b976006-ec01-4329-9666-841eee749585 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:18.572284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b976006-ec01-4329-9666-841eee749585 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:18.572284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b976006-ec01-4329-9666-841eee749585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:18.590919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-1b976006-ec01-4329-9666-841eee749585 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:18.592135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1b976006-ec01-4329-9666-841eee749585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:18.601126 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 703/1384] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:18.642899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2165d-94e6-4cef-b587-60b14db21929 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:18.658124 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2165d-94e6-4cef-b587-60b14db21929 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:18.663740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea1989-ec9c-49e0-9eba-5a087de150ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:18.664387 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea1989-ec9c-49e0-9eba-5a087de150ef service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:18.664791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea1989-ec9c-49e0-9eba-5a087de150ef service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:18.665067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea1989-ec9c-49e0-9eba-5a087de150ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:18.673887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea1989-ec9c-49e0-9eba-5a087de150ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:18.675577 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea1989-ec9c-49e0-9eba-5a087de150ef service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:18.685806 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 682/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:20.705640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4129e74-969f-482c-b925-19fa8cfe7edb None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:20.706318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4129e74-969f-482c-b925-19fa8cfe7edb None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:20.706318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4129e74-969f-482c-b925-19fa8cfe7edb None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:20.707712 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 704/1386] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:20.713095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c5dbd-22e8-4dc6-a006-7ae37c6e5a7e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:20.713574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c5dbd-22e8-4dc6-a006-7ae37c6e5a7e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:20.713671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c5dbd-22e8-4dc6-a006-7ae37c6e5a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:20.731633 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d33c5dbd-22e8-4dc6-a006-7ae37c6e5a7e None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:20.732852 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d33c5dbd-22e8-4dc6-a006-7ae37c6e5a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:20.742687 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 683/1387] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:20.787799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4129e74-969f-482c-b925-19fa8cfe7edb None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:20.806994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4129e74-969f-482c-b925-19fa8cfe7edb None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:20.814359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ac573-6509-410e-9afb-9d749affd957 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:20.815207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ac573-6509-410e-9afb-9d749affd957 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:20.815650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ac573-6509-410e-9afb-9d749affd957 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:20.815942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ac573-6509-410e-9afb-9d749affd957 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:20.824748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ac573-6509-410e-9afb-9d749affd957 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:20.826422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ac573-6509-410e-9afb-9d749affd957 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:20.836059 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 705/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:22.902747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c652d-41ab-4395-90f9-c8a9f06a9cf3 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:22.903325 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c652d-41ab-4395-90f9-c8a9f06a9cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:22.903325 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c652d-41ab-4395-90f9-c8a9f06a9cf3 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:22.904807 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 684/1389] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:22.910341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f047fcb-d713-4f3a-ab86-0a2ba46b4531 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:22.910947 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f047fcb-d713-4f3a-ab86-0a2ba46b4531 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:22.911140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f047fcb-d713-4f3a-ab86-0a2ba46b4531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:22.932743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-2f047fcb-d713-4f3a-ab86-0a2ba46b4531 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:22.934018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2f047fcb-d713-4f3a-ab86-0a2ba46b4531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:22.942443 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 706/1390] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:22.989279 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c652d-41ab-4395-90f9-c8a9f06a9cf3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:23.005030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c652d-41ab-4395-90f9-c8a9f06a9cf3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:23.011134 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2514adc-4f80-4db1-98c0-95411da023b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:23.011755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2514adc-4f80-4db1-98c0-95411da023b6 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:23.012037 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2514adc-4f80-4db1-98c0-95411da023b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:23.012316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2514adc-4f80-4db1-98c0-95411da023b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:23.020926 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2514adc-4f80-4db1-98c0-95411da023b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:23.022604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2514adc-4f80-4db1-98c0-95411da023b6 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:23.032352 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 685/1391] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:25.112941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41917e77-9298-4120-a2d3-c91009b46f89 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:25.113526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41917e77-9298-4120-a2d3-c91009b46f89 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:25.113526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41917e77-9298-4120-a2d3-c91009b46f89 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:25.117963 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 707/1392] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:25.122081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305edee-fa0d-4b44-899d-eb7a7addf087 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:25.122763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305edee-fa0d-4b44-899d-eb7a7addf087 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:25.122763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305edee-fa0d-4b44-899d-eb7a7addf087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:25.144520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1305edee-fa0d-4b44-899d-eb7a7addf087 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:25.145799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1305edee-fa0d-4b44-899d-eb7a7addf087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:25.154988 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 686/1393] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:25.201355 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41917e77-9298-4120-a2d3-c91009b46f89 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:25.220015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41917e77-9298-4120-a2d3-c91009b46f89 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:25.227197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd19fe7-d2f9-4e51-ae6b-3d654940ff21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:25.228037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd19fe7-d2f9-4e51-ae6b-3d654940ff21 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:25.228346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd19fe7-d2f9-4e51-ae6b-3d654940ff21 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:25.228657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd19fe7-d2f9-4e51-ae6b-3d654940ff21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:25.238288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd19fe7-d2f9-4e51-ae6b-3d654940ff21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:25.239958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd19fe7-d2f9-4e51-ae6b-3d654940ff21 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:25.250337 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 708/1394] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:27.351998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da413f-5e61-4089-9f9c-e00401c3402f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:27.352698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da413f-5e61-4089-9f9c-e00401c3402f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:27.352698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da413f-5e61-4089-9f9c-e00401c3402f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:27.353963 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 687/1395] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:27.359687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dab59-8a56-464f-9579-48cf7ce4dff4 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:27.360153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dab59-8a56-464f-9579-48cf7ce4dff4 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:27.360385 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dab59-8a56-464f-9579-48cf7ce4dff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:27.382608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-035dab59-8a56-464f-9579-48cf7ce4dff4 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:27.383831 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-035dab59-8a56-464f-9579-48cf7ce4dff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:27.392235 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 709/1396] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:27.436501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da413f-5e61-4089-9f9c-e00401c3402f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:27.452621 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da413f-5e61-4089-9f9c-e00401c3402f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:27.458647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95967367-481e-4a10-8bcf-c6c1d5889b92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:27.459313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95967367-481e-4a10-8bcf-c6c1d5889b92 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:27.459593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95967367-481e-4a10-8bcf-c6c1d5889b92 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:27.459907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95967367-481e-4a10-8bcf-c6c1d5889b92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:27.469572 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95967367-481e-4a10-8bcf-c6c1d5889b92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:27.471271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95967367-481e-4a10-8bcf-c6c1d5889b92 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:27.481035 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 688/1397] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:28.241807 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d0d88-38cf-4538-80d2-7e1704843483 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:28.242039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d0d88-38cf-4538-80d2-7e1704843483 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:28.242258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d0d88-38cf-4538-80d2-7e1704843483 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:28.243679 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 710/1398] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:28.249082 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24678970-1af2-4254-9537-03d8d2233c21 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:28.249370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24678970-1af2-4254-9537-03d8d2233c21 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:28.249647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24678970-1af2-4254-9537-03d8d2233c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:28.268061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-24678970-1af2-4254-9537-03d8d2233c21 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:28.269259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-24678970-1af2-4254-9537-03d8d2233c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:28.277216 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 689/1399] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:28.333195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb28fcd-157a-45a4-8863-3bf09a96fe9f None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:28.333500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb28fcd-157a-45a4-8863-3bf09a96fe9f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:28.333781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb28fcd-157a-45a4-8863-3bf09a96fe9f None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:28.335267 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 711/1400] 10.222.0.16 () {54 vars in 962 bytes} [Tue Jul 5 10:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:28.340675 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a44905-eff0-41a3-8273-c58beb5cd8d3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:28.340987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a44905-eff0-41a3-8273-c58beb5cd8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:28.341232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a44905-eff0-41a3-8273-c58beb5cd8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:28.391882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-20a44905-eff0-41a3-8273-c58beb5cd8d3 None None] MFA Rules not processed for user `b97736e49e2a4bfb9cf7093cfa8b2b16`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:28.393102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-20a44905-eff0-41a3-8273-c58beb5cd8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:28.454913 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 690/1401] 10.222.0.16 () {58 vars in 1059 bytes} [Tue Jul 5 10:05:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:28.462897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb28fcd-157a-45a4-8863-3bf09a96fe9f None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:28.474108 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb28fcd-157a-45a4-8863-3bf09a96fe9f None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:28.477669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51030d1c-a494-45b7-9166-1c320d7d5b1e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b97736e49e2a4bfb9cf7093cfa8b2b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:28.478345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51030d1c-a494-45b7-9166-1c320d7d5b1e service glance] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:28.478633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51030d1c-a494-45b7-9166-1c320d7d5b1e service glance] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:28.478912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51030d1c-a494-45b7-9166-1c320d7d5b1e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:28.488139 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51030d1c-a494-45b7-9166-1c320d7d5b1e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:28.489626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51030d1c-a494-45b7-9166-1c320d7d5b1e service glance] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:28.499555 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 712/1402] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:05:29.216062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659f152-3db2-4565-a406-050e1529c288 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:29.216587 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659f152-3db2-4565-a406-050e1529c288 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:29.216867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659f152-3db2-4565-a406-050e1529c288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:29.266993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6659f152-3db2-4565-a406-050e1529c288 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:29.268517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6659f152-3db2-4565-a406-050e1529c288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:29.309132 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:05:29.335103 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 691/1403] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:05:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:29.376034 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e580fe-f163-4437-87b5-3b718c5a33a3 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:29.376330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e580fe-f163-4437-87b5-3b718c5a33a3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:29.376619 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e580fe-f163-4437-87b5-3b718c5a33a3 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:29.378232 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 713/1404] 10.222.0.16 () {54 vars in 942 bytes} [Tue Jul 5 10:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:29.384807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953807cf-0f4d-41f3-bd3f-f1ff93adae86 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:29.385098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953807cf-0f4d-41f3-bd3f-f1ff93adae86 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:29.385378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953807cf-0f4d-41f3-bd3f-f1ff93adae86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:29.433362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-953807cf-0f4d-41f3-bd3f-f1ff93adae86 None None] MFA Rules not processed for user `dbfded29b70140a387f7b8c07c296861`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:29.434880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-953807cf-0f4d-41f3-bd3f-f1ff93adae86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:29.509497 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 692/1405] 10.222.0.16 () {58 vars in 1039 bytes} [Tue Jul 5 10:05:29 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:29.520114 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e580fe-f163-4437-87b5-3b718c5a33a3 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:29.523627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4360c27-78a0-4664-a7bc-6713e5e2e634 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:29.523926 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4360c27-78a0-4664-a7bc-6713e5e2e634 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:29.524209 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4360c27-78a0-4664-a7bc-6713e5e2e634 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:29.525605 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 693/1406] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:05:29.530912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44edee-d301-46a0-9cfa-8d41b83ddbda None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:29.531263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44edee-d301-46a0-9cfa-8d41b83ddbda None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:29.531548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44edee-d301-46a0-9cfa-8d41b83ddbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:29.534610 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e580fe-f163-4437-87b5-3b718c5a33a3 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:29.539567 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a93ca6-89d2-4bd2-b997-0175acdd4705 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:29.540212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a93ca6-89d2-4bd2-b997-0175acdd4705 service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:29.540499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a93ca6-89d2-4bd2-b997-0175acdd4705 service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:29.540775 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a93ca6-89d2-4bd2-b997-0175acdd4705 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:29.549366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a93ca6-89d2-4bd2-b997-0175acdd4705 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:29.550685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6a44edee-d301-46a0-9cfa-8d41b83ddbda None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:29.550989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a93ca6-89d2-4bd2-b997-0175acdd4705 service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:29.551950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6a44edee-d301-46a0-9cfa-8d41b83ddbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:29.561366 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 694/1408] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:29.561515 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 714/1408] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:05:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:05:29.606181 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44edee-d301-46a0-9cfa-8d41b83ddbda None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:29.618982 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44edee-d301-46a0-9cfa-8d41b83ddbda None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:29.624518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72713d61-9b62-4ceb-b881-4e199b4fa4f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:29.625301 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72713d61-9b62-4ceb-b881-4e199b4fa4f2 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:29.625608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72713d61-9b62-4ceb-b881-4e199b4fa4f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:29.625932 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72713d61-9b62-4ceb-b881-4e199b4fa4f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:29.635246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72713d61-9b62-4ceb-b881-4e199b4fa4f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:29.637405 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72713d61-9b62-4ceb-b881-4e199b4fa4f2 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:29.647308 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 695/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:39.557578 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=66 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:05:39.642536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=62 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:05:51.401044 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937252a2-3654-4c22-8f61-34989e58903a None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:51.401632 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937252a2-3654-4c22-8f61-34989e58903a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:51.401632 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937252a2-3654-4c22-8f61-34989e58903a None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:51.403065 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 715/1410] 10.222.0.16 () {54 vars in 918 bytes} [Tue Jul 5 10:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:51.409366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f964eba-8a49-4f15-8214-aeb16086adb5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:51.409992 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f964eba-8a49-4f15-8214-aeb16086adb5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:51.410195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f964eba-8a49-4f15-8214-aeb16086adb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:51.434449 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6f964eba-8a49-4f15-8214-aeb16086adb5 None None] MFA Rules not processed for user `e9d4a50f842a48b1875900426c6a1ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:51.435660 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6f964eba-8a49-4f15-8214-aeb16086adb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:51.445777 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 696/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Jul 5 10:05:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:51.460809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732bb8c-78ee-4d2b-854e-11fb2c931475 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:51.461129 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732bb8c-78ee-4d2b-854e-11fb2c931475 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:51.461462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732bb8c-78ee-4d2b-854e-11fb2c931475 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:51.463562 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 716/1412] 10.222.0.16 () {54 vars in 918 bytes} [Tue Jul 5 10:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:51.470120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0999de-c184-4627-998b-d758545aee1a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:51.470445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0999de-c184-4627-998b-d758545aee1a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:51.470722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0999de-c184-4627-998b-d758545aee1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:51.489863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4e0999de-c184-4627-998b-d758545aee1a None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:51.491089 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4e0999de-c184-4627-998b-d758545aee1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:51.501827 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 697/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Jul 5 10:05:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:53.996002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c912bc7-d0dd-4db2-9ca4-718508f20f31 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:53.996882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c912bc7-d0dd-4db2-9ca4-718508f20f31 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:53.996882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c912bc7-d0dd-4db2-9ca4-718508f20f31 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:53.999239 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:05:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:54.004760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2da5e3-8042-4bcb-a16d-b5b1a8bf26c6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:54.005227 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2da5e3-8042-4bcb-a16d-b5b1a8bf26c6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:54.005322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2da5e3-8042-4bcb-a16d-b5b1a8bf26c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:54.024493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6a2da5e3-8042-4bcb-a16d-b5b1a8bf26c6 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:54.025637 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6a2da5e3-8042-4bcb-a16d-b5b1a8bf26c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:54.034725 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:54.080848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c912bc7-d0dd-4db2-9ca4-718508f20f31 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:54.098158 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c912bc7-d0dd-4db2-9ca4-718508f20f31 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:54.105025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b836b-0196-4ebf-b954-2b78745fb077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:54.105966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b836b-0196-4ebf-b954-2b78745fb077 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:54.106544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b836b-0196-4ebf-b954-2b78745fb077 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:54.107026 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b836b-0196-4ebf-b954-2b78745fb077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:54.117896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b836b-0196-4ebf-b954-2b78745fb077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:54.119610 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b836b-0196-4ebf-b954-2b78745fb077 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:54.129639 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 718/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:55.046353 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4a4b8-32e6-4826-ad97-a61df5628580 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:55.046921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4a4b8-32e6-4826-ad97-a61df5628580 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:55.046921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4a4b8-32e6-4826-ad97-a61df5628580 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:55.048292 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:55.053426 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67057-29c7-4ae9-8721-de16925958eb None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:55.053903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67057-29c7-4ae9-8721-de16925958eb None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:55.054002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67057-29c7-4ae9-8721-de16925958eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:55.077352 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b7e67057-29c7-4ae9-8721-de16925958eb None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:55.079520 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b7e67057-29c7-4ae9-8721-de16925958eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:55.089204 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:55.245293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4a4b8-32e6-4826-ad97-a61df5628580 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:55.261774 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4a4b8-32e6-4826-ad97-a61df5628580 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:55.267586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b01b02-eae8-46bb-b00a-622a8b07f206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:55.268223 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b01b02-eae8-46bb-b00a-622a8b07f206 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:55.268501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b01b02-eae8-46bb-b00a-622a8b07f206 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:55.268774 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b01b02-eae8-46bb-b00a-622a8b07f206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:55.276716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b01b02-eae8-46bb-b00a-622a8b07f206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:55.278324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b01b02-eae8-46bb-b00a-622a8b07f206 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:55.287424 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 700/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:56.185838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebd16e-5fe4-426d-84c9-420e1bc80b97 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:56.186763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebd16e-5fe4-426d-84c9-420e1bc80b97 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:56.186763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebd16e-5fe4-426d-84c9-420e1bc80b97 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:56.188784 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:05:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:56.194829 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37ce7c-f8f4-4a26-82e8-336d8d0ced81 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:56.195554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37ce7c-f8f4-4a26-82e8-336d8d0ced81 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:56.195780 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37ce7c-f8f4-4a26-82e8-336d8d0ced81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:56.226477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6a37ce7c-f8f4-4a26-82e8-336d8d0ced81 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:56.227678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6a37ce7c-f8f4-4a26-82e8-336d8d0ced81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:56.240204 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:56.398733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebd16e-5fe4-426d-84c9-420e1bc80b97 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:56.422737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebd16e-5fe4-426d-84c9-420e1bc80b97 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:56.429727 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb64a0d-d983-40b2-890d-5b6ddb3fc259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:56.430421 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb64a0d-d983-40b2-890d-5b6ddb3fc259 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:56.430706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb64a0d-d983-40b2-890d-5b6ddb3fc259 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:56.430987 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb64a0d-d983-40b2-890d-5b6ddb3fc259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:56.440730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb64a0d-d983-40b2-890d-5b6ddb3fc259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:56.442459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb64a0d-d983-40b2-890d-5b6ddb3fc259 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:56.453061 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 721/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:57.326554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f637-2b06-44de-80eb-8541499b5b06 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:57.327400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f637-2b06-44de-80eb-8541499b5b06 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:57.327400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f637-2b06-44de-80eb-8541499b5b06 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:57.329386 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:05:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:57.336127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b92aea-ae59-42f6-8593-0d146471f5d0 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:57.336868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b92aea-ae59-42f6-8593-0d146471f5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:57.337029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b92aea-ae59-42f6-8593-0d146471f5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:57.364660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-d7b92aea-ae59-42f6-8593-0d146471f5d0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:57.365871 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d7b92aea-ae59-42f6-8593-0d146471f5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:57.375309 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:57.531648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1f637-2b06-44de-80eb-8541499b5b06 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:57.547718 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1f637-2b06-44de-80eb-8541499b5b06 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:57.553699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abec07c-ab53-44d0-9336-99d06ffa81cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:57.554434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abec07c-ab53-44d0-9336-99d06ffa81cb service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:57.554726 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abec07c-ab53-44d0-9336-99d06ffa81cb service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:57.555030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abec07c-ab53-44d0-9336-99d06ffa81cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:57.563978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abec07c-ab53-44d0-9336-99d06ffa81cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:57.565575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abec07c-ab53-44d0-9336-99d06ffa81cb service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:57.574926 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 703/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:58.448571 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1a459-c2a4-4f06-9ebb-4a264d0eb40a None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:58.449640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1a459-c2a4-4f06-9ebb-4a264d0eb40a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:58.449640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1a459-c2a4-4f06-9ebb-4a264d0eb40a None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:58.451701 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:05:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:58.460262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e07b2-c4bf-4321-90d4-36d09f738c67 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:58.460696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e07b2-c4bf-4321-90d4-36d09f738c67 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:58.460797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e07b2-c4bf-4321-90d4-36d09f738c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:58.484071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ee4e07b2-c4bf-4321-90d4-36d09f738c67 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:58.485298 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ee4e07b2-c4bf-4321-90d4-36d09f738c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:58.494355 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:58.647111 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1a459-c2a4-4f06-9ebb-4a264d0eb40a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:58.666473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1a459-c2a4-4f06-9ebb-4a264d0eb40a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:58.676147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f765dc-c316-41d3-9972-12a859aa3f6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:58.676795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f765dc-c316-41d3-9972-12a859aa3f6f service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:58.677068 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f765dc-c316-41d3-9972-12a859aa3f6f service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:58.677358 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f765dc-c316-41d3-9972-12a859aa3f6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:58.686753 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f765dc-c316-41d3-9972-12a859aa3f6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:58.688364 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f765dc-c316-41d3-9972-12a859aa3f6f service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:58.698373 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 724/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:05:59.498940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd9ba5-aaab-4fa8-92ef-2cdbb724af78 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:59.499532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd9ba5-aaab-4fa8-92ef-2cdbb724af78 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:59.499532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd9ba5-aaab-4fa8-92ef-2cdbb724af78 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:59.500914 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:05:59.506069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207b18c-3a51-4dc1-9ad9-8da0221e62af None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:59.506568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207b18c-3a51-4dc1-9ad9-8da0221e62af None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:59.506660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207b18c-3a51-4dc1-9ad9-8da0221e62af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:59.524608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e207b18c-3a51-4dc1-9ad9-8da0221e62af None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:05:59.525847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e207b18c-3a51-4dc1-9ad9-8da0221e62af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:05:59.533597 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:05:59.682179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd9ba5-aaab-4fa8-92ef-2cdbb724af78 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:05:59.706083 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd9ba5-aaab-4fa8-92ef-2cdbb724af78 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:05:59.712578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7dbec7-3902-429c-8d19-b5ea1bf88256 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:05:59.713215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7dbec7-3902-429c-8d19-b5ea1bf88256 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:05:59.713496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7dbec7-3902-429c-8d19-b5ea1bf88256 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:05:59.713771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7dbec7-3902-429c-8d19-b5ea1bf88256 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:05:59.722776 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7dbec7-3902-429c-8d19-b5ea1bf88256 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:05:59.724401 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7dbec7-3902-429c-8d19-b5ea1bf88256 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:05:59.734238 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 706/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:00.523846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb38030-644c-469b-8bae-25cb146b726f None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:00.524716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb38030-644c-469b-8bae-25cb146b726f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:00.524716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb38030-644c-469b-8bae-25cb146b726f None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:00.526839 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:06:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:00.534166 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03d93a-5115-45bb-a0d5-56ff5a2a93ab None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:00.534628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03d93a-5115-45bb-a0d5-56ff5a2a93ab None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:00.534730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03d93a-5115-45bb-a0d5-56ff5a2a93ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:00.555136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fb03d93a-5115-45bb-a0d5-56ff5a2a93ab None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:00.556379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fb03d93a-5115-45bb-a0d5-56ff5a2a93ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:00.565770 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:00.720333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb38030-644c-469b-8bae-25cb146b726f None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:00.738178 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb38030-644c-469b-8bae-25cb146b726f None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:00.749040 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00afc7-d89e-4f1b-bed1-aa757f54aa57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:00.749721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00afc7-d89e-4f1b-bed1-aa757f54aa57 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:00.750013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00afc7-d89e-4f1b-bed1-aa757f54aa57 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:00.750281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00afc7-d89e-4f1b-bed1-aa757f54aa57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:00.760863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00afc7-d89e-4f1b-bed1-aa757f54aa57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:00.762544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00afc7-d89e-4f1b-bed1-aa757f54aa57 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:00.772804 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 727/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:01.583413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50f0c-70c8-4350-9b38-b635cf8c0e70 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:01.584324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50f0c-70c8-4350-9b38-b635cf8c0e70 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:01.584324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50f0c-70c8-4350-9b38-b635cf8c0e70 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:01.587214 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:06:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:01.592417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caff62-d293-4cac-9fff-983fdb88f02a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:01.592897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caff62-d293-4cac-9fff-983fdb88f02a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:01.593007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caff62-d293-4cac-9fff-983fdb88f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:01.616958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-37caff62-d293-4cac-9fff-983fdb88f02a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:01.618173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-37caff62-d293-4cac-9fff-983fdb88f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:01.627601 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:01.779476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f50f0c-70c8-4350-9b38-b635cf8c0e70 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:01.801450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f50f0c-70c8-4350-9b38-b635cf8c0e70 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:01.807188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a969d-cc4e-4836-b896-197d8fd63159 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:01.808192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a969d-cc4e-4836-b896-197d8fd63159 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:01.808474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a969d-cc4e-4836-b896-197d8fd63159 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:01.809259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a969d-cc4e-4836-b896-197d8fd63159 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:01.817855 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a969d-cc4e-4836-b896-197d8fd63159 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:01.819472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a969d-cc4e-4836-b896-197d8fd63159 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:01.830277 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 709/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:02.623406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55300e84-fb63-4447-8aae-8c0fa240d2c8 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:02.624016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55300e84-fb63-4447-8aae-8c0fa240d2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:02.624016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55300e84-fb63-4447-8aae-8c0fa240d2c8 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:02.625359 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:02.631666 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21553f-df0c-481e-bb3d-92deec1477e2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:02.632403 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21553f-df0c-481e-bb3d-92deec1477e2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:02.632577 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21553f-df0c-481e-bb3d-92deec1477e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:02.656225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4a21553f-df0c-481e-bb3d-92deec1477e2 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:02.657504 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4a21553f-df0c-481e-bb3d-92deec1477e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:02.666765 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:02.815407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55300e84-fb63-4447-8aae-8c0fa240d2c8 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:02.830394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55300e84-fb63-4447-8aae-8c0fa240d2c8 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:02.837110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64275b22-f0b0-47be-b84c-8da95b72b135 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:02.837780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64275b22-f0b0-47be-b84c-8da95b72b135 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:02.838061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64275b22-f0b0-47be-b84c-8da95b72b135 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:02.838372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64275b22-f0b0-47be-b84c-8da95b72b135 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:02.846783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64275b22-f0b0-47be-b84c-8da95b72b135 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:02.848382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64275b22-f0b0-47be-b84c-8da95b72b135 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:02.857768 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 730/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:04.775127 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c4bd2-73d4-41d6-9c05-482e10134e28 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:04.775625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c4bd2-73d4-41d6-9c05-482e10134e28 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:04.775720 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c4bd2-73d4-41d6-9c05-482e10134e28 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:04.777085 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:04.782904 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3113e2-3264-47d5-a258-40b9891c8431 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:04.783327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3113e2-3264-47d5-a258-40b9891c8431 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:04.783467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3113e2-3264-47d5-a258-40b9891c8431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:04.801814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-df3113e2-3264-47d5-a258-40b9891c8431 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:04.803067 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-df3113e2-3264-47d5-a258-40b9891c8431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:04.812241 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:04.855520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c4bd2-73d4-41d6-9c05-482e10134e28 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:04.858789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5d5c4bd2-73d4-41d6-9c05-482e10134e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:04.874739 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c4bd2-73d4-41d6-9c05-482e10134e28 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:04.878318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e203a1b-2e76-4651-a85f-27193da27c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:04.878949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e203a1b-2e76-4651-a85f-27193da27c45 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:04.879230 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e203a1b-2e76-4651-a85f-27193da27c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:04.879508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e203a1b-2e76-4651-a85f-27193da27c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:04.888376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e203a1b-2e76-4651-a85f-27193da27c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:04.889789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e203a1b-2e76-4651-a85f-27193da27c45 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:04.899415 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 712/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:07.237871 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be981f-b22f-46c1-b47b-db47626ef67d None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:07.238716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be981f-b22f-46c1-b47b-db47626ef67d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:07.238716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be981f-b22f-46c1-b47b-db47626ef67d None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:07.240814 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:06:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:07.246732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d06e14-2c7f-480a-944a-7880f4f24c07 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:07.247213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d06e14-2c7f-480a-944a-7880f4f24c07 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:07.247321 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d06e14-2c7f-480a-944a-7880f4f24c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:07.265784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-33d06e14-2c7f-480a-944a-7880f4f24c07 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:07.267022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-33d06e14-2c7f-480a-944a-7880f4f24c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:07.275128 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:07.313389 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be981f-b22f-46c1-b47b-db47626ef67d None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:07.326731 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be981f-b22f-46c1-b47b-db47626ef67d None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:07.331132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0392a-3547-491c-8f9e-57e25313c8d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:07.331790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0392a-3547-491c-8f9e-57e25313c8d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:07.332072 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0392a-3547-491c-8f9e-57e25313c8d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:07.332344 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0392a-3547-491c-8f9e-57e25313c8d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:07.342978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0392a-3547-491c-8f9e-57e25313c8d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:07.344411 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0392a-3547-491c-8f9e-57e25313c8d2 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:07.355947 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 733/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:09.727218 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d41f27-a95f-44c8-84c4-0515faf191c0 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:09.728138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d41f27-a95f-44c8-84c4-0515faf191c0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:09.728138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d41f27-a95f-44c8-84c4-0515faf191c0 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:09.730317 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:06:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:09.739504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f955-4b9b-4fb0-9a8a-7ab2fdd7cac8 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:09.739957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f955-4b9b-4fb0-9a8a-7ab2fdd7cac8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:09.740062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f955-4b9b-4fb0-9a8a-7ab2fdd7cac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:09.764237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-9d77f955-4b9b-4fb0-9a8a-7ab2fdd7cac8 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:09.765467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9d77f955-4b9b-4fb0-9a8a-7ab2fdd7cac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:09.774361 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:09.813441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d41f27-a95f-44c8-84c4-0515faf191c0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:09.825827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d41f27-a95f-44c8-84c4-0515faf191c0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:09.829786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d52fa-14b1-412e-a0cc-185c9919a41a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:09.830467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d52fa-14b1-412e-a0cc-185c9919a41a service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:09.830754 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d52fa-14b1-412e-a0cc-185c9919a41a service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:09.831033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d52fa-14b1-412e-a0cc-185c9919a41a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:09.840434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d52fa-14b1-412e-a0cc-185c9919a41a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:09.841867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d52fa-14b1-412e-a0cc-185c9919a41a service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:09.851269 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 715/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:11.982911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840cf1-d8f4-402b-87f5-f132734ff3f8 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:11.983511 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840cf1-d8f4-402b-87f5-f132734ff3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:11.983511 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840cf1-d8f4-402b-87f5-f132734ff3f8 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:11.985585 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:06:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:11.994771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b751a-d822-43d1-bca8-4433caf1b25d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:11.995170 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b751a-d822-43d1-bca8-4433caf1b25d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:11.995432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b751a-d822-43d1-bca8-4433caf1b25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:12.015835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6a5b751a-d822-43d1-bca8-4433caf1b25d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:12.016940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6a5b751a-d822-43d1-bca8-4433caf1b25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:12.025228 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:12.063331 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38840cf1-d8f4-402b-87f5-f132734ff3f8 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:12.076310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38840cf1-d8f4-402b-87f5-f132734ff3f8 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:12.080815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a6f4d-0f29-45a6-9d3f-af155ae816d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:12.081462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6f4d-0f29-45a6-9d3f-af155ae816d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:12.081741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6f4d-0f29-45a6-9d3f-af155ae816d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:12.082013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6f4d-0f29-45a6-9d3f-af155ae816d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:12.091070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a6f4d-0f29-45a6-9d3f-af155ae816d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:12.092508 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a6f4d-0f29-45a6-9d3f-af155ae816d6 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:12.101717 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 736/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:14.249275 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f849bad-35d2-4706-8e19-640111e7efae None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:14.249858 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f849bad-35d2-4706-8e19-640111e7efae None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:14.249858 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f849bad-35d2-4706-8e19-640111e7efae None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:14.251294 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:14.258071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d9427-079d-4e87-bce7-49976871844c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:14.258568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d9427-079d-4e87-bce7-49976871844c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:14.258692 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d9427-079d-4e87-bce7-49976871844c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:14.295636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-505d9427-079d-4e87-bce7-49976871844c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:14.297596 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-505d9427-079d-4e87-bce7-49976871844c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:14.306856 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:14.352444 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f849bad-35d2-4706-8e19-640111e7efae None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:14.365603 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f849bad-35d2-4706-8e19-640111e7efae None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:14.369703 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87042f34-7f7e-4b2c-9ee3-6e50af58eb4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:14.370375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87042f34-7f7e-4b2c-9ee3-6e50af58eb4a service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:14.370656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87042f34-7f7e-4b2c-9ee3-6e50af58eb4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:14.370925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87042f34-7f7e-4b2c-9ee3-6e50af58eb4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:14.380942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87042f34-7f7e-4b2c-9ee3-6e50af58eb4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:14.382439 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87042f34-7f7e-4b2c-9ee3-6e50af58eb4a service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:14.392304 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 718/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:16.481287 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7316aa-a406-4d0e-8f7a-6f4c34d30f6b None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:16.481866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7316aa-a406-4d0e-8f7a-6f4c34d30f6b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:16.481866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7316aa-a406-4d0e-8f7a-6f4c34d30f6b None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:16.483284 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:16.488647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b9fa6-950f-4549-85c7-24cdbd7a22ac None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:16.489117 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b9fa6-950f-4549-85c7-24cdbd7a22ac None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:16.489229 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b9fa6-950f-4549-85c7-24cdbd7a22ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:16.508110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c85b9fa6-950f-4549-85c7-24cdbd7a22ac None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:16.509369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c85b9fa6-950f-4549-85c7-24cdbd7a22ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:16.517539 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:16.557797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7316aa-a406-4d0e-8f7a-6f4c34d30f6b None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:16.574906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7316aa-a406-4d0e-8f7a-6f4c34d30f6b None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:16.581078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b811b-b943-4e13-9bd8-365cc992196c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:16.581994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b811b-b943-4e13-9bd8-365cc992196c service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:16.582352 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b811b-b943-4e13-9bd8-365cc992196c service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:16.582691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b811b-b943-4e13-9bd8-365cc992196c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:16.593532 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b811b-b943-4e13-9bd8-365cc992196c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:16.595012 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b811b-b943-4e13-9bd8-365cc992196c service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:16.604784 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 739/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:06:18.655582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15184b63-a988-4386-9838-3bf1194aae96 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:18.656159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15184b63-a988-4386-9838-3bf1194aae96 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:18.656159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15184b63-a988-4386-9838-3bf1194aae96 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:18.657540 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:18.664122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0934-58ab-4f76-9d66-3cd2fe117ae4 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:18.664979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0934-58ab-4f76-9d66-3cd2fe117ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:18.665148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0934-58ab-4f76-9d66-3cd2fe117ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:18.684657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-0c7d0934-58ab-4f76-9d66-3cd2fe117ae4 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:18.685884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0c7d0934-58ab-4f76-9d66-3cd2fe117ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:18.695437 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:18.735801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15184b63-a988-4386-9838-3bf1194aae96 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:18.748829 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15184b63-a988-4386-9838-3bf1194aae96 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:18.753361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5e3f1-b998-4ac3-a378-7c2de916e3fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:18.754036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e3f1-b998-4ac3-a378-7c2de916e3fb service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:18.754350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e3f1-b998-4ac3-a378-7c2de916e3fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:18.754628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e3f1-b998-4ac3-a378-7c2de916e3fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:18.766306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5e3f1-b998-4ac3-a378-7c2de916e3fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:18.767799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5e3f1-b998-4ac3-a378-7c2de916e3fb service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:18.777907 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 721/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:06:20.822383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80cc53-b384-4bd3-ab74-d7decc80d13c None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:20.823280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80cc53-b384-4bd3-ab74-d7decc80d13c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:20.823454 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80cc53-b384-4bd3-ab74-d7decc80d13c None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:20.825610 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:06:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:20.834793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e311c-39ad-43f3-80df-fb91fa631fb6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:20.835209 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e311c-39ad-43f3-80df-fb91fa631fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:20.835393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e311c-39ad-43f3-80df-fb91fa631fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:20.846010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=79 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:06:20.860986 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d72e311c-39ad-43f3-80df-fb91fa631fb6 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:20.862270 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d72e311c-39ad-43f3-80df-fb91fa631fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:20.871533 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:20.908654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80cc53-b384-4bd3-ab74-d7decc80d13c None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:20.914752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=73 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:06:20.921234 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80cc53-b384-4bd3-ab74-d7decc80d13c None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:20.925227 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba25c4-62aa-4b55-806e-4010a88611a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:20.925877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba25c4-62aa-4b55-806e-4010a88611a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:20.926155 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba25c4-62aa-4b55-806e-4010a88611a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:20.926524 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba25c4-62aa-4b55-806e-4010a88611a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:20.935863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba25c4-62aa-4b55-806e-4010a88611a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:20.937335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba25c4-62aa-4b55-806e-4010a88611a4 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:20.947376 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 742/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:22.954123 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5e327-6138-464b-b30b-7eb328230023 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:22.954734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5e327-6138-464b-b30b-7eb328230023 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:22.954734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5e327-6138-464b-b30b-7eb328230023 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:22.956088 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:22.962631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02506-d633-4a85-9774-3889ca6205c4 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:22.963107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02506-d633-4a85-9774-3889ca6205c4 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:22.963199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02506-d633-4a85-9774-3889ca6205c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:22.985665 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-91c02506-d633-4a85-9774-3889ca6205c4 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:22.986899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-91c02506-d633-4a85-9774-3889ca6205c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:22.995242 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:23.031128 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5e327-6138-464b-b30b-7eb328230023 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:23.042680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5e327-6138-464b-b30b-7eb328230023 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:23.046582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ce4c4-b646-4d26-9a4c-258d78cb82af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:23.047221 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ce4c4-b646-4d26-9a4c-258d78cb82af service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:23.047504 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ce4c4-b646-4d26-9a4c-258d78cb82af service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:23.047772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ce4c4-b646-4d26-9a4c-258d78cb82af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:23.056974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ce4c4-b646-4d26-9a4c-258d78cb82af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:23.058456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ce4c4-b646-4d26-9a4c-258d78cb82af service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:23.068114 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 724/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:06:25.108808 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f61f21-72f2-4c62-9793-b52afa3d7665 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:25.109624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f61f21-72f2-4c62-9793-b52afa3d7665 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:25.109624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f61f21-72f2-4c62-9793-b52afa3d7665 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:25.111982 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:06:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:25.117607 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53970ef4-64b1-40d3-b627-f10fb0e9c821 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:25.118069 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53970ef4-64b1-40d3-b627-f10fb0e9c821 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:25.118175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53970ef4-64b1-40d3-b627-f10fb0e9c821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:25.136788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-53970ef4-64b1-40d3-b627-f10fb0e9c821 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:25.137990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-53970ef4-64b1-40d3-b627-f10fb0e9c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:25.146498 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:25.185797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f61f21-72f2-4c62-9793-b52afa3d7665 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:25.202435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f61f21-72f2-4c62-9793-b52afa3d7665 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:25.206752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f7f2-c00e-4a27-a30c-88e93eb28263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:25.207524 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f7f2-c00e-4a27-a30c-88e93eb28263 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:25.207818 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f7f2-c00e-4a27-a30c-88e93eb28263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:25.208126 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f7f2-c00e-4a27-a30c-88e93eb28263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:25.218140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242f7f2-c00e-4a27-a30c-88e93eb28263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:25.219611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242f7f2-c00e-4a27-a30c-88e93eb28263 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:25.229478 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 745/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:35.131641 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=11 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:06:35.224890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=14 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:06:45.014772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3b3d5-ec9d-4b27-988b-bea555c16f98 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:45.015515 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3b3d5-ec9d-4b27-988b-bea555c16f98 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:45.015515 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3b3d5-ec9d-4b27-988b-bea555c16f98 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:45.016859 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 726/1471] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:45.022600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b34ba-df46-4ae4-92bc-2139368916cb None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:45.023051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b34ba-df46-4ae4-92bc-2139368916cb None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:45.023157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b34ba-df46-4ae4-92bc-2139368916cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:45.042936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b33b34ba-df46-4ae4-92bc-2139368916cb None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:45.044151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b33b34ba-df46-4ae4-92bc-2139368916cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:45.053400 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 746/1472] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:45.140351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e62401-df6b-48e1-a273-a3eca9ec332a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:45.140640 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e62401-df6b-48e1-a273-a3eca9ec332a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:45.140917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e62401-df6b-48e1-a273-a3eca9ec332a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:45.142366 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 727/1473] 10.222.0.16 () {54 vars in 962 bytes} [Tue Jul 5 10:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:45.147353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353afdd-cc16-4d5f-a959-628a3393eacc None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:45.147646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353afdd-cc16-4d5f-a959-628a3393eacc None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:45.147922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353afdd-cc16-4d5f-a959-628a3393eacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:45.166872 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-f353afdd-cc16-4d5f-a959-628a3393eacc None None] MFA Rules not processed for user `b97736e49e2a4bfb9cf7093cfa8b2b16`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:45.168071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f353afdd-cc16-4d5f-a959-628a3393eacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:45.174875 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 747/1474] 10.222.0.16 () {58 vars in 1059 bytes} [Tue Jul 5 10:06:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:45.182309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e62401-df6b-48e1-a273-a3eca9ec332a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:45.195851 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e62401-df6b-48e1-a273-a3eca9ec332a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:45.199665 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c203bb-a1a4-4f56-9468-3c8c0a652820 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b97736e49e2a4bfb9cf7093cfa8b2b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:45.200291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c203bb-a1a4-4f56-9468-3c8c0a652820 service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:45.200608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c203bb-a1a4-4f56-9468-3c8c0a652820 service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:45.200873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c203bb-a1a4-4f56-9468-3c8c0a652820 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:45.210418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c203bb-a1a4-4f56-9468-3c8c0a652820 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:45.212646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c203bb-a1a4-4f56-9468-3c8c0a652820 service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:45.223104 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 728/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:47.321894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbee9c-e00e-485b-9ee8-f120478eb541 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:47.322537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbee9c-e00e-485b-9ee8-f120478eb541 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:47.322537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbee9c-e00e-485b-9ee8-f120478eb541 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:47.323914 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 748/1476] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:06:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:47.330121 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa2af5-71f6-4dae-8c2d-05ead627c30a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:47.330940 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa2af5-71f6-4dae-8c2d-05ead627c30a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:47.331076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa2af5-71f6-4dae-8c2d-05ead627c30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:47.354750 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-50fa2af5-71f6-4dae-8c2d-05ead627c30a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:47.355950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-50fa2af5-71f6-4dae-8c2d-05ead627c30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:47.364265 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 729/1477] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:47.387611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbee9c-e00e-485b-9ee8-f120478eb541 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:47.402709 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbee9c-e00e-485b-9ee8-f120478eb541 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:47.408311 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461645be-88c4-4e67-b564-5034c6ba89e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:47.408958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461645be-88c4-4e67-b564-5034c6ba89e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:47.409242 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461645be-88c4-4e67-b564-5034c6ba89e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:47.409512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461645be-88c4-4e67-b564-5034c6ba89e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:47.410267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461645be-88c4-4e67-b564-5034c6ba89e2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:47.412053 dv-cinder-master-3-smb devstack@keystone.service[80376]: WARNING py.warnings [None req-461645be-88c4-4e67-b564-5034c6ba89e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:06:47.412053 dv-cinder-master-3-smb devstack@keystone.service[80376]: warnings.warn(msg) Jul 05 10:06:47.412053 dv-cinder-master-3-smb devstack@keystone.service[80376]: Jul 05 10:06:47.413181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461645be-88c4-4e67-b564-5034c6ba89e2 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:47.432500 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 749/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jul 5 10:06:47 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:06:49.447837 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfffef-5369-4ec0-b95a-339fb063df05 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:49.448468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfffef-5369-4ec0-b95a-339fb063df05 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:49.448468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfffef-5369-4ec0-b95a-339fb063df05 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:49.449773 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 730/1479] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:49.456283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c169c-acbd-4b0d-acf9-74505c027959 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:49.457044 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c169c-acbd-4b0d-acf9-74505c027959 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:49.457155 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c169c-acbd-4b0d-acf9-74505c027959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:49.485976 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-bf8c169c-acbd-4b0d-acf9-74505c027959 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:49.487269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-bf8c169c-acbd-4b0d-acf9-74505c027959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:49.496315 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 750/1480] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:49.540673 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfffef-5369-4ec0-b95a-339fb063df05 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:49.558130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfffef-5369-4ec0-b95a-339fb063df05 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:49.564659 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ff90a-6bf0-44c1-b6ca-6d901024f9ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:49.565293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ff90a-6bf0-44c1-b6ca-6d901024f9ba service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:49.565576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ff90a-6bf0-44c1-b6ca-6d901024f9ba service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:49.565847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ff90a-6bf0-44c1-b6ca-6d901024f9ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:49.574578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ff90a-6bf0-44c1-b6ca-6d901024f9ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:49.576151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ff90a-6bf0-44c1-b6ca-6d901024f9ba service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:49.585625 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 731/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:51.784875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4206e65-ab95-4b28-b0d7-6ad90911bb47 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:51.785354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4206e65-ab95-4b28-b0d7-6ad90911bb47 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:51.785447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4206e65-ab95-4b28-b0d7-6ad90911bb47 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:51.787624 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 751/1482] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:06:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:51.793600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a1779-d896-4c9e-8bd2-950df4d47c09 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:51.794046 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a1779-d896-4c9e-8bd2-950df4d47c09 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:51.794202 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a1779-d896-4c9e-8bd2-950df4d47c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:51.819727 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-044a1779-d896-4c9e-8bd2-950df4d47c09 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:51.820945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-044a1779-d896-4c9e-8bd2-950df4d47c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:51.830249 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 732/1483] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:51.871627 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4206e65-ab95-4b28-b0d7-6ad90911bb47 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:51.885536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4206e65-ab95-4b28-b0d7-6ad90911bb47 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:51.889700 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfb392-2513-4903-b7f0-27536c22c50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:51.890395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfb392-2513-4903-b7f0-27536c22c50f service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:51.890686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfb392-2513-4903-b7f0-27536c22c50f service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:51.890967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfb392-2513-4903-b7f0-27536c22c50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:51.900839 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfb392-2513-4903-b7f0-27536c22c50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:51.902345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfb392-2513-4903-b7f0-27536c22c50f service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:51.912156 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 752/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:53.971827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674c339-386b-4e8b-8f01-741ded791d28 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:53.972389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674c339-386b-4e8b-8f01-741ded791d28 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:53.972389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674c339-386b-4e8b-8f01-741ded791d28 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:53.974698 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 733/1485] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:06:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:53.983380 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fcf35-14e6-4b95-bfd4-36d3c89bf713 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:53.983839 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fcf35-14e6-4b95-bfd4-36d3c89bf713 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:53.983988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fcf35-14e6-4b95-bfd4-36d3c89bf713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:54.003538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-c76fcf35-14e6-4b95-bfd4-36d3c89bf713 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:54.004754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c76fcf35-14e6-4b95-bfd4-36d3c89bf713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:54.014039 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 753/1486] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:54.055330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674c339-386b-4e8b-8f01-741ded791d28 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:54.070468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674c339-386b-4e8b-8f01-741ded791d28 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:54.075336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb18f3-811a-498e-b4c5-e5a981a7b061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:54.075976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb18f3-811a-498e-b4c5-e5a981a7b061 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:54.076257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb18f3-811a-498e-b4c5-e5a981a7b061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:54.076529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb18f3-811a-498e-b4c5-e5a981a7b061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:54.090636 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb18f3-811a-498e-b4c5-e5a981a7b061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:54.092079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb18f3-811a-498e-b4c5-e5a981a7b061 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:54.101920 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 734/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:56.200360 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453a80-96aa-4484-bfae-8cd6cf72a874 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:56.201245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453a80-96aa-4484-bfae-8cd6cf72a874 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:56.201245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453a80-96aa-4484-bfae-8cd6cf72a874 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:56.203503 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 754/1488] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:06:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:56.208878 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adf275-b2dc-4741-a866-0745d7864dfd None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:56.209327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adf275-b2dc-4741-a866-0745d7864dfd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:56.209422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adf275-b2dc-4741-a866-0745d7864dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:56.227478 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-66adf275-b2dc-4741-a866-0745d7864dfd None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:56.228705 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-66adf275-b2dc-4741-a866-0745d7864dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:56.236830 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 735/1489] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:56.280281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6453a80-96aa-4484-bfae-8cd6cf72a874 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:56.292202 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6453a80-96aa-4484-bfae-8cd6cf72a874 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:56.296002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4062a94-44fc-46bb-a9d9-267e6cb2c756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:56.296646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4062a94-44fc-46bb-a9d9-267e6cb2c756 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:56.296926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4062a94-44fc-46bb-a9d9-267e6cb2c756 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:56.297200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4062a94-44fc-46bb-a9d9-267e6cb2c756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:56.306327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4062a94-44fc-46bb-a9d9-267e6cb2c756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:56.307876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4062a94-44fc-46bb-a9d9-267e6cb2c756 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:56.317307 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 755/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:06:58.625562 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f5db3-aa3b-4a98-8f13-bd064c1cf89d None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:58.626183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f5db3-aa3b-4a98-8f13-bd064c1cf89d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:58.626183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f5db3-aa3b-4a98-8f13-bd064c1cf89d None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:58.628229 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 736/1491] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jul 5 10:06:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:06:58.633837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf7ed9-fe21-4d89-93d4-13b784348540 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:58.634334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf7ed9-fe21-4d89-93d4-13b784348540 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:58.634458 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf7ed9-fe21-4d89-93d4-13b784348540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:58.653063 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ebcf7ed9-fe21-4d89-93d4-13b784348540 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:06:58.654324 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ebcf7ed9-fe21-4d89-93d4-13b784348540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:06:58.663170 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 756/1492] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jul 5 10:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:06:58.705880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f5db3-aa3b-4a98-8f13-bd064c1cf89d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:06:58.718495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f5db3-aa3b-4a98-8f13-bd064c1cf89d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:06:58.722888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76c5c6-365a-48e3-9753-e15014d69f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:06:58.723553 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c5c6-365a-48e3-9753-e15014d69f26 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:06:58.723840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c5c6-365a-48e3-9753-e15014d69f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:06:58.724116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c5c6-365a-48e3-9753-e15014d69f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:06:58.733379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76c5c6-365a-48e3-9753-e15014d69f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:06:58.734847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76c5c6-365a-48e3-9753-e15014d69f26 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:06:58.744378 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 737/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:07:08.647743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=33 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:07:08.740216 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=33 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:08:24.728743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23273e11-d65d-4872-a2cd-461ca49684c3 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:24.729682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23273e11-d65d-4872-a2cd-461ca49684c3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:24.729682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23273e11-d65d-4872-a2cd-461ca49684c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:24.755706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-23273e11-d65d-4872-a2cd-461ca49684c3 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:08:24.756918 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-23273e11-d65d-4872-a2cd-461ca49684c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:08:24.766484 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 757/1494] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:08:24.774752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df877ea9-b65e-45a2-8ef9-fd30a0fa019a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:24.775242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df877ea9-b65e-45a2-8ef9-fd30a0fa019a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:24.775359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df877ea9-b65e-45a2-8ef9-fd30a0fa019a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:24.794749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-df877ea9-b65e-45a2-8ef9-fd30a0fa019a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:08:24.795959 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-df877ea9-b65e-45a2-8ef9-fd30a0fa019a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:08:24.807746 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 738/1495] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:08:24.812501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23273e11-d65d-4872-a2cd-461ca49684c3 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:08:24.822718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23273e11-d65d-4872-a2cd-461ca49684c3 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:08:24.828813 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052e523-da10-48ee-b32b-4833c1901c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:08:24.829470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052e523-da10-48ee-b32b-4833c1901c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:24.829758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052e523-da10-48ee-b32b-4833c1901c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:24.830029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052e523-da10-48ee-b32b-4833c1901c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:24.830838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052e523-da10-48ee-b32b-4833c1901c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:08:24.831142 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052e523-da10-48ee-b32b-4833c1901c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:08:24.832191 dv-cinder-master-3-smb devstack@keystone.service[80376]: WARNING py.warnings [None req-6052e523-da10-48ee-b32b-4833c1901c58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:24.832191 dv-cinder-master-3-smb devstack@keystone.service[80376]: warnings.warn(msg) Jul 05 10:08:24.832191 dv-cinder-master-3-smb devstack@keystone.service[80376]: Jul 05 10:08:24.833069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052e523-da10-48ee-b32b-4833c1901c58 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:08:24.845316 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 758/1496] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:08:24.853373 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df877ea9-b65e-45a2-8ef9-fd30a0fa019a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:08:24.863179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df877ea9-b65e-45a2-8ef9-fd30a0fa019a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:08:24.869364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4635a-522c-4f9b-bea8-be61c9518c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:08:24.870019 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4635a-522c-4f9b-bea8-be61c9518c94 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:24.870336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4635a-522c-4f9b-bea8-be61c9518c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:24.870618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4635a-522c-4f9b-bea8-be61c9518c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:24.871318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4635a-522c-4f9b-bea8-be61c9518c94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:08:24.873085 dv-cinder-master-3-smb devstack@keystone.service[80377]: WARNING py.warnings [None req-41d4635a-522c-4f9b-bea8-be61c9518c94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:24.873085 dv-cinder-master-3-smb devstack@keystone.service[80377]: warnings.warn(msg) Jul 05 10:08:24.873085 dv-cinder-master-3-smb devstack@keystone.service[80377]: Jul 05 10:08:24.874324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4635a-522c-4f9b-bea8-be61c9518c94 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:08:24.915047 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 739/1497] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:08:24 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:08:24.919354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052e523-da10-48ee-b32b-4833c1901c58 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:08:24.929687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052e523-da10-48ee-b32b-4833c1901c58 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:08:24.986106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa57677-dfbd-44c8-bf5d-421ee8ec5cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:08:24.986734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa57677-dfbd-44c8-bf5d-421ee8ec5cad admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:24.987029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa57677-dfbd-44c8-bf5d-421ee8ec5cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:24.987298 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa57677-dfbd-44c8-bf5d-421ee8ec5cad admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:24.987982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa57677-dfbd-44c8-bf5d-421ee8ec5cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:08:24.989007 dv-cinder-master-3-smb devstack@keystone.service[80376]: WARNING py.warnings [None req-eaa57677-dfbd-44c8-bf5d-421ee8ec5cad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:24.989007 dv-cinder-master-3-smb devstack@keystone.service[80376]: warnings.warn(msg) Jul 05 10:08:24.989007 dv-cinder-master-3-smb devstack@keystone.service[80376]: Jul 05 10:08:24.990212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa57677-dfbd-44c8-bf5d-421ee8ec5cad admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:08:25.015180 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 759/1498] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:08:24 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:08:25.019423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4635a-522c-4f9b-bea8-be61c9518c94 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:08:25.030186 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4635a-522c-4f9b-bea8-be61c9518c94 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:08:25.035208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb6924-e996-4d7a-b09d-395a6a55c601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:08:25.035841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb6924-e996-4d7a-b09d-395a6a55c601 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:25.036160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb6924-e996-4d7a-b09d-395a6a55c601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:25.036437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb6924-e996-4d7a-b09d-395a6a55c601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:25.037174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb6924-e996-4d7a-b09d-395a6a55c601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:08:25.038194 dv-cinder-master-3-smb devstack@keystone.service[80377]: WARNING py.warnings [None req-65bb6924-e996-4d7a-b09d-395a6a55c601 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:25.038194 dv-cinder-master-3-smb devstack@keystone.service[80377]: warnings.warn(msg) Jul 05 10:08:25.038194 dv-cinder-master-3-smb devstack@keystone.service[80377]: Jul 05 10:08:25.039125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb6924-e996-4d7a-b09d-395a6a55c601 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:08:25.052005 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 740/1499] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:08:25.057776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa57677-dfbd-44c8-bf5d-421ee8ec5cad admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:08:25.072812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa57677-dfbd-44c8-bf5d-421ee8ec5cad admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:08:25.078580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b62b3-ec01-4423-9c5a-1a3d2945e00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:08:25.079262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b62b3-ec01-4423-9c5a-1a3d2945e00b admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:25.079542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b62b3-ec01-4423-9c5a-1a3d2945e00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:25.079816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b62b3-ec01-4423-9c5a-1a3d2945e00b admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:25.080529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b62b3-ec01-4423-9c5a-1a3d2945e00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:08:25.081504 dv-cinder-master-3-smb devstack@keystone.service[80376]: WARNING py.warnings [None req-942b62b3-ec01-4423-9c5a-1a3d2945e00b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:25.081504 dv-cinder-master-3-smb devstack@keystone.service[80376]: warnings.warn(msg) Jul 05 10:08:25.081504 dv-cinder-master-3-smb devstack@keystone.service[80376]: Jul 05 10:08:25.082408 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b62b3-ec01-4423-9c5a-1a3d2945e00b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:08:25.096119 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 760/1500] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:08:25.100248 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb6924-e996-4d7a-b09d-395a6a55c601 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:08:25.111012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb6924-e996-4d7a-b09d-395a6a55c601 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:08:25.116472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93883e4c-f518-4b00-b7f2-a2f454c4b8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:08:25.117356 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93883e4c-f518-4b00-b7f2-a2f454c4b8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:25.117638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93883e4c-f518-4b00-b7f2-a2f454c4b8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:25.117912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93883e4c-f518-4b00-b7f2-a2f454c4b8bf admin admin] PATH_INFO: `/v3/projects/c557f69c11774472ae2764637fec79b0/users/69aea82d6cee4567803786319c0f39f9/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:25.134573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93883e4c-f518-4b00-b7f2-a2f454c4b8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c557f69c11774472ae2764637fec79b0, user_id=69aea82d6cee4567803786319c0f39f9, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:08:25.135598 dv-cinder-master-3-smb devstack@keystone.service[80377]: WARNING py.warnings [None req-93883e4c-f518-4b00-b7f2-a2f454c4b8bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:25.135598 dv-cinder-master-3-smb devstack@keystone.service[80377]: warnings.warn(msg) Jul 05 10:08:25.135598 dv-cinder-master-3-smb devstack@keystone.service[80377]: Jul 05 10:08:25.137616 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93883e4c-f518-4b00-b7f2-a2f454c4b8bf admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:08:25.158813 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 741/1501] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:08:25 2022] PUT /identity/v3/projects/c557f69c11774472ae2764637fec79b0/users/69aea82d6cee4567803786319c0f39f9/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:08:25.174544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f7a3-e37c-47c3-b677-8f42e474762d None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:25.174840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f7a3-e37c-47c3-b677-8f42e474762d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:25.175105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f7a3-e37c-47c3-b677-8f42e474762d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:25.204124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-4575f7a3-e37c-47c3-b677-8f42e474762d None None] MFA Rules not processed for user `69aea82d6cee4567803786319c0f39f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:08:25.205351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4575f7a3-e37c-47c3-b677-8f42e474762d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:08:25.273472 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 761/1502] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:08:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:08:25.287315 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8ff38-aee1-4529-b7d6-27d80a0f31fc None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:25.287635 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8ff38-aee1-4529-b7d6-27d80a0f31fc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:25.287962 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8ff38-aee1-4529-b7d6-27d80a0f31fc None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:25.289452 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 742/1503] 10.222.0.16 () {54 vars in 963 bytes} [Tue Jul 5 10:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:08:25.294479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778f82b-8df5-4c28-8551-3278368e823c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:25.294777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778f82b-8df5-4c28-8551-3278368e823c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:25.295054 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778f82b-8df5-4c28-8551-3278368e823c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:25.313889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e778f82b-8df5-4c28-8551-3278368e823c None None] MFA Rules not processed for user `dc11037d649241608ea1378cf4692dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:08:25.315135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e778f82b-8df5-4c28-8551-3278368e823c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:08:25.359814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:08:25.368634 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 762/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Tue Jul 5 10:08:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:08:25.376548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8ff38-aee1-4529-b7d6-27d80a0f31fc None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:08:25.388451 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8ff38-aee1-4529-b7d6-27d80a0f31fc None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:08:25.392759 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3d2ab-053f-4647-87f1-2b1609c7f2cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:08:25.393471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3d2ab-053f-4647-87f1-2b1609c7f2cd service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:25.393767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3d2ab-053f-4647-87f1-2b1609c7f2cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:25.394044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3d2ab-053f-4647-87f1-2b1609c7f2cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:25.402757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3d2ab-053f-4647-87f1-2b1609c7f2cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:08:25.404261 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3d2ab-053f-4647-87f1-2b1609c7f2cd service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:08:25.413362 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 743/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:08:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:08:25.458177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888ee94-834c-47a4-8f4f-c15e2693e50b None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:25.458547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888ee94-834c-47a4-8f4f-c15e2693e50b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:25.458890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888ee94-834c-47a4-8f4f-c15e2693e50b None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:25.461175 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 763/1506] 10.222.0.16 () {52 vars in 829 bytes} [Tue Jul 5 10:08:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:08:25.465175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3d2ab-053f-4647-87f1-2b1609c7f2cd service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:08:25.539656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3d2ab-053f-4647-87f1-2b1609c7f2cd service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:08:25.544588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:08:25.545471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:25.545771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:25.546047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] PATH_INFO: `/v3/users/69aea82d6cee4567803786319c0f39f9` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:25.547294 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69aea82d6cee4567803786319c0f39f9)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:08:25.548309 dv-cinder-master-3-smb devstack@keystone.service[80377]: WARNING py.warnings [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:25.548309 dv-cinder-master-3-smb devstack@keystone.service[80377]: warnings.warn(msg) Jul 05 10:08:25.548309 dv-cinder-master-3-smb devstack@keystone.service[80377]: Jul 05 10:08:25.549505 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:08:25.614701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69aea82d6cee4567803786319c0f39f9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:08:25.615521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69aea82d6cee4567803786319c0f39f9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:08:25.616231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69aea82d6cee4567803786319c0f39f9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:08:25.630025 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69aea82d6cee4567803786319c0f39f9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:08:25.650166 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69aea82d6cee4567803786319c0f39f9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:08:25.654490 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:08:25.668455 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 744/1507] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:08:25 2022] DELETE /identity/v3/users/69aea82d6cee4567803786319c0f39f9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:08:25.673099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffcaf6-fa10-43e8-bd18-837a6eccf86d None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:25.673394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffcaf6-fa10-43e8-bd18-837a6eccf86d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:25.673677 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffcaf6-fa10-43e8-bd18-837a6eccf86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:25.691624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e6ffcaf6-fa10-43e8-bd18-837a6eccf86d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:08:25.692835 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e6ffcaf6-fa10-43e8-bd18-837a6eccf86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:08:25.757578 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 764/1508] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:08:25.767522 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:08:25.769093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:08:25.832283 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9f8a2-d21a-4dc1-acc0-0c21885aef97 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:08:25.836511 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750923f-e53c-4e61-a1dd-8da5fd01ad32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:08:25.837190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750923f-e53c-4e61-a1dd-8da5fd01ad32 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:25.837473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750923f-e53c-4e61-a1dd-8da5fd01ad32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:25.837751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750923f-e53c-4e61-a1dd-8da5fd01ad32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:25.846986 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750923f-e53c-4e61-a1dd-8da5fd01ad32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:08:25.848436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750923f-e53c-4e61-a1dd-8da5fd01ad32 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:08:25.857856 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 745/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:08:26.239222 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffcaf6-fa10-43e8-bd18-837a6eccf86d None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:08:26.240747 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e6ffcaf6-fa10-43e8-bd18-837a6eccf86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:08:26.253392 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffcaf6-fa10-43e8-bd18-837a6eccf86d None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:08:26.258092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:08:26.258998 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:08:26.259281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:08:26.259552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] PATH_INFO: `/v3/projects/c557f69c11774472ae2764637fec79b0` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:08:26.260776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c557f69c11774472ae2764637fec79b0)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:08:26.261799 dv-cinder-master-3-smb devstack@keystone.service[80376]: WARNING py.warnings [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:26.261799 dv-cinder-master-3-smb devstack@keystone.service[80376]: warnings.warn(msg) Jul 05 10:08:26.261799 dv-cinder-master-3-smb devstack@keystone.service[80376]: Jul 05 10:08:26.263022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:08:26.323594 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c557f69c11774472ae2764637fec79b0', 'request_id': 'req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:08:26.329514 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c557f69c11774472ae2764637fec79b0', 'request_id': 'req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:08:26.342759 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c557f69c11774472ae2764637fec79b0', 'request_id': 'req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:08:26.357433 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] The token cache is being invalidate because project c557f69c11774472ae2764637fec79b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:08:26.357649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:08:26.358399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-bb8b9e0d-5ef8-4acc-a1e1-820738d373cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:08:26.359952 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 765/1510] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:08:26 2022] DELETE /identity/v3/projects/c557f69c11774472ae2764637fec79b0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:08:35.853666 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:08:36.354387 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:14.590986 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4a491-760c-4bc2-9da5-fb2868c8149a None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:14.592004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4a491-760c-4bc2-9da5-fb2868c8149a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:14.592004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4a491-760c-4bc2-9da5-fb2868c8149a None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:14.593902 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:10:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:10:14.600719 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcb83a-2949-48de-ace8-017dec45d25b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:14.601201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcb83a-2949-48de-ace8-017dec45d25b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:14.601305 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcb83a-2949-48de-ace8-017dec45d25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:14.626778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-9cdcb83a-2949-48de-ace8-017dec45d25b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:10:14.627972 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9cdcb83a-2949-48de-ace8-017dec45d25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:14.687608 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:10:14.843587 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4a491-760c-4bc2-9da5-fb2868c8149a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:10:14.846724 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2bb4a491-760c-4bc2-9da5-fb2868c8149a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:14.930572 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4a491-760c-4bc2-9da5-fb2868c8149a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:10:14.936600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f61e1-d54f-4e01-85c7-41800ec22859 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:10:14.937258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f61e1-d54f-4e01-85c7-41800ec22859 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:14.937542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f61e1-d54f-4e01-85c7-41800ec22859 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:14.937819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f61e1-d54f-4e01-85c7-41800ec22859 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:14.946259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f61e1-d54f-4e01-85c7-41800ec22859 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:10:14.947884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f61e1-d54f-4e01-85c7-41800ec22859 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:10:14.957296 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 747/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:10:15.777330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735761e-f06c-4f82-aa28-a581c34ca18c None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:15.777798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735761e-f06c-4f82-aa28-a581c34ca18c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:15.777903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735761e-f06c-4f82-aa28-a581c34ca18c None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:15.779316 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:10:15.785068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aec99-a4b8-4641-9449-e6341fce28b0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:15.785520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aec99-a4b8-4641-9449-e6341fce28b0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:15.785621 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aec99-a4b8-4641-9449-e6341fce28b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:15.804789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-966aec99-a4b8-4641-9449-e6341fce28b0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:10:15.806040 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-966aec99-a4b8-4641-9449-e6341fce28b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:15.815290 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:10:15.973133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735761e-f06c-4f82-aa28-a581c34ca18c None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:10:16.000647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735761e-f06c-4f82-aa28-a581c34ca18c None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:10:16.010968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ed1b5-d9ad-4c03-9284-3088284fbe42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:10:16.012028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ed1b5-d9ad-4c03-9284-3088284fbe42 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:16.012321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ed1b5-d9ad-4c03-9284-3088284fbe42 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:16.012611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ed1b5-d9ad-4c03-9284-3088284fbe42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:16.021955 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ed1b5-d9ad-4c03-9284-3088284fbe42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:10:16.023599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ed1b5-d9ad-4c03-9284-3088284fbe42 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:10:16.034257 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 768/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:10:16.542320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6184f38-b3ae-4232-9265-7b127ff91b79 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:16.542549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6184f38-b3ae-4232-9265-7b127ff91b79 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:16.542833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6184f38-b3ae-4232-9265-7b127ff91b79 None None] PATH_INFO: `/v3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:16.544164 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 749/1517] 10.222.0.39 () {54 vars in 925 bytes} [Tue Jul 5 10:10:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:10:16.550719 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5d910-fba5-4794-a93a-ffd5e8dd8105 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:16.550988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5d910-fba5-4794-a93a-ffd5e8dd8105 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:16.551312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5d910-fba5-4794-a93a-ffd5e8dd8105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:16.571046 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e2f5d910-fba5-4794-a93a-ffd5e8dd8105 None None] MFA Rules not processed for user `e9d4a50f842a48b1875900426c6a1ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:10:16.572285 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e2f5d910-fba5-4794-a93a-ffd5e8dd8105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:16.640019 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 769/1518] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Jul 5 10:10:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:10:25.799172 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=37 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:25.882411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e67fb-ab97-4630-8331-6d136d7b76bf None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:25.882600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e67fb-ab97-4630-8331-6d136d7b76bf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:25.882844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e67fb-ab97-4630-8331-6d136d7b76bf None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:25.885077 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 750/1519] 10.222.0.16 () {54 vars in 960 bytes} [Tue Jul 5 10:10:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:10:25.893447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b04b9-f3c0-4bc2-b515-281d3cf855a7 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:25.893923 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b04b9-f3c0-4bc2-b515-281d3cf855a7 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:25.894176 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b04b9-f3c0-4bc2-b515-281d3cf855a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:25.915229 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-359b04b9-f3c0-4bc2-b515-281d3cf855a7 None None] MFA Rules not processed for user `e9d4a50f842a48b1875900426c6a1ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:10:25.916437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-359b04b9-f3c0-4bc2-b515-281d3cf855a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:25.927338 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 770/1520] 10.222.0.16 () {58 vars in 1057 bytes} [Tue Jul 5 10:10:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:10:25.935019 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e67fb-ab97-4630-8331-6d136d7b76bf None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:10:25.949439 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e67fb-ab97-4630-8331-6d136d7b76bf None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:10:25.955693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f0a40-2682-4b6c-b0ca-51d3c6e3b992 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:10:25.956321 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f0a40-2682-4b6c-b0ca-51d3c6e3b992 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:25.956597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f0a40-2682-4b6c-b0ca-51d3c6e3b992 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:25.956876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f0a40-2682-4b6c-b0ca-51d3c6e3b992 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:25.965852 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f0a40-2682-4b6c-b0ca-51d3c6e3b992 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:10:25.967456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f0a40-2682-4b6c-b0ca-51d3c6e3b992 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:10:25.977356 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 751/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:10:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:10:27.265626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77b016-09c2-480e-972b-277a158c315a None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:27.266260 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77b016-09c2-480e-972b-277a158c315a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:27.266260 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77b016-09c2-480e-972b-277a158c315a None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:27.267627 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:10:27.272760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c65789-b1e3-44d8-a669-eb3754fb6f86 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:27.273213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c65789-b1e3-44d8-a669-eb3754fb6f86 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:27.273410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c65789-b1e3-44d8-a669-eb3754fb6f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:27.292925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d3c65789-b1e3-44d8-a669-eb3754fb6f86 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:10:27.294128 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d3c65789-b1e3-44d8-a669-eb3754fb6f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:27.303151 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:10:27.458914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77b016-09c2-480e-972b-277a158c315a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:10:27.475374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77b016-09c2-480e-972b-277a158c315a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:10:27.481366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc89aac-b4b3-4492-a800-f1dfcf4824d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:10:27.482012 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc89aac-b4b3-4492-a800-f1dfcf4824d7 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:27.482321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc89aac-b4b3-4492-a800-f1dfcf4824d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:27.482600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc89aac-b4b3-4492-a800-f1dfcf4824d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:27.494489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc89aac-b4b3-4492-a800-f1dfcf4824d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:10:27.497011 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc89aac-b4b3-4492-a800-f1dfcf4824d7 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:10:27.512077 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 772/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:10:28.331656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8236fa-00ca-4069-98e0-65ed4eef0693 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:28.332254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8236fa-00ca-4069-98e0-65ed4eef0693 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:28.332254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8236fa-00ca-4069-98e0-65ed4eef0693 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:28.333643 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 10:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:10:28.340071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f9358-6fd1-42ef-8f24-3269cb7ab0d3 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:28.340781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f9358-6fd1-42ef-8f24-3269cb7ab0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:28.340891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f9358-6fd1-42ef-8f24-3269cb7ab0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:28.368699 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-9c0f9358-6fd1-42ef-8f24-3269cb7ab0d3 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:10:28.369916 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9c0f9358-6fd1-42ef-8f24-3269cb7ab0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:28.379755 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 10:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:10:28.534518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8236fa-00ca-4069-98e0-65ed4eef0693 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:10:28.551227 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8236fa-00ca-4069-98e0-65ed4eef0693 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:10:28.556776 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfbbb1-a874-48d8-b5eb-5c3700ff80fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:10:28.557421 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfbbb1-a874-48d8-b5eb-5c3700ff80fd service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:28.557706 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfbbb1-a874-48d8-b5eb-5c3700ff80fd service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:28.558011 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfbbb1-a874-48d8-b5eb-5c3700ff80fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:28.566620 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfbbb1-a874-48d8-b5eb-5c3700ff80fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:10:28.568307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfbbb1-a874-48d8-b5eb-5c3700ff80fd service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:10:28.578375 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 754/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:10:32.823963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae8acc-30c9-4b82-a0b8-014d64647d8d None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:32.824538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae8acc-30c9-4b82-a0b8-014d64647d8d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:32.824538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae8acc-30c9-4b82-a0b8-014d64647d8d None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:32.825905 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:10:32.832271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace60013-a5b9-4a41-923b-122ac44532b5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:32.832722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace60013-a5b9-4a41-923b-122ac44532b5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:32.832835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace60013-a5b9-4a41-923b-122ac44532b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:32.853016 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ace60013-a5b9-4a41-923b-122ac44532b5 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:10:32.854263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ace60013-a5b9-4a41-923b-122ac44532b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:32.862747 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:10:32.941849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae8acc-30c9-4b82-a0b8-014d64647d8d None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:10:32.944078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-71ae8acc-30c9-4b82-a0b8-014d64647d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:33.009403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae8acc-30c9-4b82-a0b8-014d64647d8d None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:10:33.013299 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79c950-9a90-47b9-9b92-323b40a68dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:10:33.013970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79c950-9a90-47b9-9b92-323b40a68dae service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:33.014253 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79c950-9a90-47b9-9b92-323b40a68dae service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:33.014554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79c950-9a90-47b9-9b92-323b40a68dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:33.024876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79c950-9a90-47b9-9b92-323b40a68dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:10:33.026350 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79c950-9a90-47b9-9b92-323b40a68dae service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:10:33.031845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:33.037730 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 775/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:10:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:10:35.103686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50be1f-683f-4e1e-9ddc-c806c6bcda11 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:35.104281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50be1f-683f-4e1e-9ddc-c806c6bcda11 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:35.104281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50be1f-683f-4e1e-9ddc-c806c6bcda11 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:35.105538 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:10:35.111023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519f71d-3b16-45c7-bb5e-7d79cd1710e0 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:35.111483 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519f71d-3b16-45c7-bb5e-7d79cd1710e0 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:35.111587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519f71d-3b16-45c7-bb5e-7d79cd1710e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:35.131514 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-d519f71d-3b16-45c7-bb5e-7d79cd1710e0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:10:35.132686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d519f71d-3b16-45c7-bb5e-7d79cd1710e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:35.141679 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:10:35.180680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50be1f-683f-4e1e-9ddc-c806c6bcda11 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:10:35.197153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50be1f-683f-4e1e-9ddc-c806c6bcda11 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:10:35.201589 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f76f7-93fc-4c92-8d51-c99f0911a234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:10:35.202242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f76f7-93fc-4c92-8d51-c99f0911a234 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:35.202546 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f76f7-93fc-4c92-8d51-c99f0911a234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:35.202826 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f76f7-93fc-4c92-8d51-c99f0911a234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:35.212133 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f76f7-93fc-4c92-8d51-c99f0911a234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:10:35.213955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f76f7-93fc-4c92-8d51-c99f0911a234 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:10:35.223430 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 757/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:10:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:10:38.761710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dcf96-d2ac-4422-a7ae-b84c6a5f5c4a None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:38.762211 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dcf96-d2ac-4422-a7ae-b84c6a5f5c4a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:38.762211 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dcf96-d2ac-4422-a7ae-b84c6a5f5c4a None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:38.763730 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:10:38.770071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39174fa-06e9-4fb8-b36c-c4eb37ce7353 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:38.770883 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39174fa-06e9-4fb8-b36c-c4eb37ce7353 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:38.771039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39174fa-06e9-4fb8-b36c-c4eb37ce7353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:38.792300 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b39174fa-06e9-4fb8-b36c-c4eb37ce7353 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:10:38.793502 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b39174fa-06e9-4fb8-b36c-c4eb37ce7353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:38.802121 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:10:39.108342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193dcf96-d2ac-4422-a7ae-b84c6a5f5c4a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:10:39.111283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-193dcf96-d2ac-4422-a7ae-b84c6a5f5c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:39.181198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193dcf96-d2ac-4422-a7ae-b84c6a5f5c4a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:10:39.187730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335f349-2c0b-4679-a61c-fe1886c902c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:10:39.188556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335f349-2c0b-4679-a61c-fe1886c902c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:39.188891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335f349-2c0b-4679-a61c-fe1886c902c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:39.189240 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335f349-2c0b-4679-a61c-fe1886c902c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:39.200329 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335f349-2c0b-4679-a61c-fe1886c902c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:10:39.201826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335f349-2c0b-4679-a61c-fe1886c902c3 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:10:39.211533 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 778/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:10:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:10:41.264027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7712a-6c60-41d9-813a-4d4e9763b26f None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:41.264618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7712a-6c60-41d9-813a-4d4e9763b26f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:41.264618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7712a-6c60-41d9-813a-4d4e9763b26f None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:41.266014 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 10:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:10:41.272481 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3939-154f-4677-903e-c51b325907c3 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:41.273242 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3939-154f-4677-903e-c51b325907c3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:41.273401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3939-154f-4677-903e-c51b325907c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:41.298007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ad4e3939-154f-4677-903e-c51b325907c3 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:10:41.299258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ad4e3939-154f-4677-903e-c51b325907c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:41.308322 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 10:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:10:41.611230 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7712a-6c60-41d9-813a-4d4e9763b26f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:10:41.613862 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e6a7712a-6c60-41d9-813a-4d4e9763b26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:10:41.631018 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7712a-6c60-41d9-813a-4d4e9763b26f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:10:41.635037 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631752c-923b-4b2b-8bb5-650f15037c8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:10:41.635676 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631752c-923b-4b2b-8bb5-650f15037c8a service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:10:41.635958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631752c-923b-4b2b-8bb5-650f15037c8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:10:41.636238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631752c-923b-4b2b-8bb5-650f15037c8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:10:41.645494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631752c-923b-4b2b-8bb5-650f15037c8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:10:41.647169 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631752c-923b-4b2b-8bb5-650f15037c8a service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:10:41.656876 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 760/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:10:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:10:51.292614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=32 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:51.652677 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=33 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:17.599027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544031f-2bc2-41af-a34c-5a9fe320a8ac None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:17.606272 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544031f-2bc2-41af-a34c-5a9fe320a8ac None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:17.606272 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544031f-2bc2-41af-a34c-5a9fe320a8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:17.623625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e544031f-2bc2-41af-a34c-5a9fe320a8ac None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:17.624852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e544031f-2bc2-41af-a34c-5a9fe320a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:17.638303 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 780/1540] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:17.647400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6c29-caff-46b1-88d5-eab2d2d787f6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:17.648007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6c29-caff-46b1-88d5-eab2d2d787f6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:17.648007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6c29-caff-46b1-88d5-eab2d2d787f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:17.675433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f92d6c29-caff-46b1-88d5-eab2d2d787f6 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:17.676980 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f92d6c29-caff-46b1-88d5-eab2d2d787f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:17.691825 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 761/1541] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:17.697087 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544031f-2bc2-41af-a34c-5a9fe320a8ac None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:17.707521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544031f-2bc2-41af-a34c-5a9fe320a8ac None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:17.718878 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa5b0b-4b24-46cb-8e74-23bdd407b63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:17.719578 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa5b0b-4b24-46cb-8e74-23bdd407b63a admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:17.719859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa5b0b-4b24-46cb-8e74-23bdd407b63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:17.720130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa5b0b-4b24-46cb-8e74-23bdd407b63a admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:17.720873 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa5b0b-4b24-46cb-8e74-23bdd407b63a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:17.721175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa5b0b-4b24-46cb-8e74-23bdd407b63a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:17.722925 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa5b0b-4b24-46cb-8e74-23bdd407b63a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:17.742637 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 781/1542] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:17.751120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d6c29-caff-46b1-88d5-eab2d2d787f6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:17.764625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d6c29-caff-46b1-88d5-eab2d2d787f6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:17.787225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfc2ff-d001-4c64-8c68-e569df61d625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:17.787963 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc2ff-d001-4c64-8c68-e569df61d625 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:17.788250 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc2ff-d001-4c64-8c68-e569df61d625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:17.788523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc2ff-d001-4c64-8c68-e569df61d625 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:17.789279 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfc2ff-d001-4c64-8c68-e569df61d625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:17.791763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfc2ff-d001-4c64-8c68-e569df61d625 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:17.856834 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 762/1543] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:17 2022] POST /identity/v3/projects => generated 403 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:17.862573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa5b0b-4b24-46cb-8e74-23bdd407b63a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:17.879373 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa5b0b-4b24-46cb-8e74-23bdd407b63a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:17.985896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612f95f-b740-4694-b10c-a995187ce369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:17.986658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612f95f-b740-4694-b10c-a995187ce369 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:17.986939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612f95f-b740-4694-b10c-a995187ce369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:17.987213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612f95f-b740-4694-b10c-a995187ce369 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:17.987913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d612f95f-b740-4694-b10c-a995187ce369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:17.989942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d612f95f-b740-4694-b10c-a995187ce369 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.020195 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 782/1544] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:17 2022] POST /identity/v3/users => generated 302 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:18.024495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfc2ff-d001-4c64-8c68-e569df61d625 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.052679 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfc2ff-d001-4c64-8c68-e569df61d625 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.061292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37c91b-8b06-4541-9fa4-2f79c0c21fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.062285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37c91b-8b06-4541-9fa4-2f79c0c21fde admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.062790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37c91b-8b06-4541-9fa4-2f79c0c21fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.063267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37c91b-8b06-4541-9fa4-2f79c0c21fde admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.064270 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37c91b-8b06-4541-9fa4-2f79c0c21fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.066130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37c91b-8b06-4541-9fa4-2f79c0c21fde admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.087252 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 763/1545] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:18.092116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612f95f-b740-4694-b10c-a995187ce369 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.104899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612f95f-b740-4694-b10c-a995187ce369 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.111617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f60a5-b14b-4d28-ae8f-08e76f166748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.112246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f60a5-b14b-4d28-ae8f-08e76f166748 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.112529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f60a5-b14b-4d28-ae8f-08e76f166748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.112804 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f60a5-b14b-4d28-ae8f-08e76f166748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.113627 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f60a5-b14b-4d28-ae8f-08e76f166748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.115357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f60a5-b14b-4d28-ae8f-08e76f166748 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.132212 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 783/1546] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:18.136456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37c91b-8b06-4541-9fa4-2f79c0c21fde admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.151613 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37c91b-8b06-4541-9fa4-2f79c0c21fde admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.171282 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d6ef2-4ee2-4a02-8dfd-0028e7074a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.172549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d6ef2-4ee2-4a02-8dfd-0028e7074a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.173045 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d6ef2-4ee2-4a02-8dfd-0028e7074a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.173508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d6ef2-4ee2-4a02-8dfd-0028e7074a8e admin admin] PATH_INFO: `/v3/projects/91a8c19c62ad4d41aca91d7edcf00591/users/a98ac3936d784bed9aa5b510100deb14/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.198391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d6ef2-4ee2-4a02-8dfd-0028e7074a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a8c19c62ad4d41aca91d7edcf00591, user_id=a98ac3936d784bed9aa5b510100deb14, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.201604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d6ef2-4ee2-4a02-8dfd-0028e7074a8e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.227894 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 764/1547] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:18 2022] PUT /identity/v3/projects/91a8c19c62ad4d41aca91d7edcf00591/users/a98ac3936d784bed9aa5b510100deb14/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:18.234083 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce50a0-12b3-4396-ae5a-4e43614eaa84 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.234530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce50a0-12b3-4396-ae5a-4e43614eaa84 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.234940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce50a0-12b3-4396-ae5a-4e43614eaa84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.257816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-f2ce50a0-12b3-4396-ae5a-4e43614eaa84 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.259265 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f2ce50a0-12b3-4396-ae5a-4e43614eaa84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.268137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41b45c-af71-42db-9ea3-517dc2cda3ba None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.268735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41b45c-af71-42db-9ea3-517dc2cda3ba None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.269206 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41b45c-af71-42db-9ea3-517dc2cda3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.300347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9d41b45c-af71-42db-9ea3-517dc2cda3ba None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.301945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9d41b45c-af71-42db-9ea3-517dc2cda3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.341397 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 784/1548] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.344388 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b113b-958e-48d1-8d6f-99c21ec0440f None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.344683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b113b-958e-48d1-8d6f-99c21ec0440f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.344956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b113b-958e-48d1-8d6f-99c21ec0440f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.388391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-685b113b-958e-48d1-8d6f-99c21ec0440f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.391121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-685b113b-958e-48d1-8d6f-99c21ec0440f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.403288 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 785/1549] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.409465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7937692-2dae-4897-8b94-92eee35f3027 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.409754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7937692-2dae-4897-8b94-92eee35f3027 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.410030 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7937692-2dae-4897-8b94-92eee35f3027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.419938 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 765/1550] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:12:18.421539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41b45c-af71-42db-9ea3-517dc2cda3ba None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.435721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-f7937692-2dae-4897-8b94-92eee35f3027 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.437092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f7937692-2dae-4897-8b94-92eee35f3027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.445236 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 786/1551] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.446360 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9935-fbb4-4fe9-b487-afc90596e219 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.446654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9935-fbb4-4fe9-b487-afc90596e219 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.446937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9935-fbb4-4fe9-b487-afc90596e219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.466803 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-0dab9935-fbb4-4fe9-b487-afc90596e219 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.468128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0dab9935-fbb4-4fe9-b487-afc90596e219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.476033 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 787/1552] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.477267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6672d-7080-4063-8161-16154d28e00a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.477532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41b45c-af71-42db-9ea3-517dc2cda3ba None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.477646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6672d-7080-4063-8161-16154d28e00a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.477819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6672d-7080-4063-8161-16154d28e00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.484588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ab507-bf4e-4770-b866-3d4bfc48375c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.485279 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ab507-bf4e-4770-b866-3d4bfc48375c service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.485566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ab507-bf4e-4770-b866-3d4bfc48375c service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.485841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ab507-bf4e-4770-b866-3d4bfc48375c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.495174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ab507-bf4e-4770-b866-3d4bfc48375c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.496681 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ab507-bf4e-4770-b866-3d4bfc48375c service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.498673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-64b6672d-7080-4063-8161-16154d28e00a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.499846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-64b6672d-7080-4063-8161-16154d28e00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.506118 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 766/1553] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:18.507479 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3e8ad-cd91-4b1f-8fb7-f7e2cee2aebc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.507619 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 788/1554] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.515541 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3e8ad-cd91-4b1f-8fb7-f7e2cee2aebc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.515541 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3e8ad-cd91-4b1f-8fb7-f7e2cee2aebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.516013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2441c-263f-4788-b328-bb052d1c0b87 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.516013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2441c-263f-4788-b328-bb052d1c0b87 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.516013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2441c-263f-4788-b328-bb052d1c0b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.527605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7ac3e8ad-cd91-4b1f-8fb7-f7e2cee2aebc None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.527957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e8f2441c-263f-4788-b328-bb052d1c0b87 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.528831 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7ac3e8ad-cd91-4b1f-8fb7-f7e2cee2aebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.529367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e8f2441c-263f-4788-b328-bb052d1c0b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.537447 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 767/1555] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.538793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9df5-bb26-46f6-b113-0796c5d488aa None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.539100 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9df5-bb26-46f6-b113-0796c5d488aa None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.539384 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9df5-bb26-46f6-b113-0796c5d488aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.545495 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 789/1556] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.546803 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f18068-f4e1-45a1-859d-5e83b0548afe None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.547103 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f18068-f4e1-45a1-859d-5e83b0548afe None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.547380 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f18068-f4e1-45a1-859d-5e83b0548afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.561509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f8cb9df5-bb26-46f6-b113-0796c5d488aa None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.562848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f8cb9df5-bb26-46f6-b113-0796c5d488aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.565328 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-49f18068-f4e1-45a1-859d-5e83b0548afe None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.566600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-49f18068-f4e1-45a1-859d-5e83b0548afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.571384 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 768/1557] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.575628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea94aa4-fee7-42fc-8395-27e58bb76bf1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.575952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea94aa4-fee7-42fc-8395-27e58bb76bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.576236 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea94aa4-fee7-42fc-8395-27e58bb76bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.582419 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 790/1558] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.583825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e71fc-0894-434e-97ff-91847d88f03b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.584144 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e71fc-0894-434e-97ff-91847d88f03b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.584421 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e71fc-0894-434e-97ff-91847d88f03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.597191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4ea94aa4-fee7-42fc-8395-27e58bb76bf1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.599311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4ea94aa4-fee7-42fc-8395-27e58bb76bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.604495 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-694e71fc-0894-434e-97ff-91847d88f03b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.605666 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-694e71fc-0894-434e-97ff-91847d88f03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.608258 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 769/1559] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.609598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296e64-5023-449c-b8a4-1c22cb37b9f2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.609935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296e64-5023-449c-b8a4-1c22cb37b9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.610242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296e64-5023-449c-b8a4-1c22cb37b9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.615194 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 791/1560] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.618798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e47f82-79e8-4237-b3e9-0d6bef85b0ed None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.618798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e47f82-79e8-4237-b3e9-0d6bef85b0ed None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.618798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e47f82-79e8-4237-b3e9-0d6bef85b0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.634058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fb296e64-5023-449c-b8a4-1c22cb37b9f2 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.635429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fb296e64-5023-449c-b8a4-1c22cb37b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.635856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-40e47f82-79e8-4237-b3e9-0d6bef85b0ed None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.637013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-40e47f82-79e8-4237-b3e9-0d6bef85b0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.643599 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 770/1561] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.644408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296e64-5023-449c-b8a4-1c22cb37b9f2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.644946 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 792/1562] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.646257 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f90a7a-57e6-46f3-ab03-ef2c8ae39005 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.646539 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f90a7a-57e6-46f3-ab03-ef2c8ae39005 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.646814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f90a7a-57e6-46f3-ab03-ef2c8ae39005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.657967 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296e64-5023-449c-b8a4-1c22cb37b9f2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.660967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:18.663266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f63e4a-ae5d-4e9c-8805-d5f3ee1a0c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.664708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f63e4a-ae5d-4e9c-8805-d5f3ee1a0c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.664887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f63e4a-ae5d-4e9c-8805-d5f3ee1a0c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.665303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f63e4a-ae5d-4e9c-8805-d5f3ee1a0c44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.666253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f63e4a-ae5d-4e9c-8805-d5f3ee1a0c44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:18.666558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f63e4a-ae5d-4e9c-8805-d5f3ee1a0c44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.667942 dv-cinder-master-3-smb devstack@keystone.service[80377]: WARNING py.warnings [None req-43f63e4a-ae5d-4e9c-8805-d5f3ee1a0c44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:12:18.667942 dv-cinder-master-3-smb devstack@keystone.service[80377]: warnings.warn(msg) Jul 05 10:12:18.667942 dv-cinder-master-3-smb devstack@keystone.service[80377]: Jul 05 10:12:18.668913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b3f90a7a-57e6-46f3-ab03-ef2c8ae39005 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:18.669187 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f63e4a-ae5d-4e9c-8805-d5f3ee1a0c44 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.671777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b3f90a7a-57e6-46f3-ab03-ef2c8ae39005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:18.681095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:18.684286 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 793/1563] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:18.685309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f90a7a-57e6-46f3-ab03-ef2c8ae39005 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.691081 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 771/1564] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:18.692712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f63e4a-ae5d-4e9c-8805-d5f3ee1a0c44 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.696338 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f90a7a-57e6-46f3-ab03-ef2c8ae39005 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.702011 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90d79f-6827-4c68-af6c-448932632cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.702631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90d79f-6827-4c68-af6c-448932632cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.702948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90d79f-6827-4c68-af6c-448932632cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.703297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90d79f-6827-4c68-af6c-448932632cd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.704020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90d79f-6827-4c68-af6c-448932632cd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:18.704353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90d79f-6827-4c68-af6c-448932632cd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.704702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f63e4a-ae5d-4e9c-8805-d5f3ee1a0c44 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.706205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90d79f-6827-4c68-af6c-448932632cd7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.712281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ce1ea-02f3-4279-af27-0dd6466c7109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.712281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ce1ea-02f3-4279-af27-0dd6466c7109 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.712281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ce1ea-02f3-4279-af27-0dd6466c7109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.712281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ce1ea-02f3-4279-af27-0dd6466c7109 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.712628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ce1ea-02f3-4279-af27-0dd6466c7109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:18.712628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ce1ea-02f3-4279-af27-0dd6466c7109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.713882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ce1ea-02f3-4279-af27-0dd6466c7109 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.722075 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 794/1565] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:18.723012 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90d79f-6827-4c68-af6c-448932632cd7 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.733773 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90d79f-6827-4c68-af6c-448932632cd7 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.739450 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee281b-4a34-4d23-b66c-2e34de19c9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.742373 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 772/1566] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:18.742528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee281b-4a34-4d23-b66c-2e34de19c9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.742617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ce1ea-02f3-4279-af27-0dd6466c7109 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.742705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee281b-4a34-4d23-b66c-2e34de19c9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.742705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee281b-4a34-4d23-b66c-2e34de19c9d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.742705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee281b-4a34-4d23-b66c-2e34de19c9d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:18.742705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee281b-4a34-4d23-b66c-2e34de19c9d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.745047 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee281b-4a34-4d23-b66c-2e34de19c9d3 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.753987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ce1ea-02f3-4279-af27-0dd6466c7109 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.759645 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 795/1567] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:18.760569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee281b-4a34-4d23-b66c-2e34de19c9d3 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.761321 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224def71-9b1c-4987-915c-96da1512b45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.762302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224def71-9b1c-4987-915c-96da1512b45a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.762798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224def71-9b1c-4987-915c-96da1512b45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.763642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224def71-9b1c-4987-915c-96da1512b45a admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.764648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224def71-9b1c-4987-915c-96da1512b45a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:18.765174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224def71-9b1c-4987-915c-96da1512b45a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.767536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224def71-9b1c-4987-915c-96da1512b45a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.775423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee281b-4a34-4d23-b66c-2e34de19c9d3 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.780901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53f7fc-e4f3-485b-9261-52686fd19432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.781599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53f7fc-e4f3-485b-9261-52686fd19432 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.781875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53f7fc-e4f3-485b-9261-52686fd19432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.782148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53f7fc-e4f3-485b-9261-52686fd19432 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.782985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53f7fc-e4f3-485b-9261-52686fd19432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:18.783292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53f7fc-e4f3-485b-9261-52686fd19432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.784900 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53f7fc-e4f3-485b-9261-52686fd19432 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.786851 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 773/1568] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:18.789196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224def71-9b1c-4987-915c-96da1512b45a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.798955 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 796/1569] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:18.799776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53f7fc-e4f3-485b-9261-52686fd19432 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.807737 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224def71-9b1c-4987-915c-96da1512b45a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.810688 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53f7fc-e4f3-485b-9261-52686fd19432 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.812572 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f554e3-01e7-4c28-a919-9b827f1a221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.813230 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f554e3-01e7-4c28-a919-9b827f1a221b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.813521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f554e3-01e7-4c28-a919-9b827f1a221b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.813790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f554e3-01e7-4c28-a919-9b827f1a221b admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.814610 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f554e3-01e7-4c28-a919-9b827f1a221b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:18.814918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f554e3-01e7-4c28-a919-9b827f1a221b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.815644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaba13-d844-453c-8a91-3256a498b2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.816326 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaba13-d844-453c-8a91-3256a498b2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.816720 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f554e3-01e7-4c28-a919-9b827f1a221b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.816825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaba13-d844-453c-8a91-3256a498b2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.816995 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaba13-d844-453c-8a91-3256a498b2b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.817688 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaba13-d844-453c-8a91-3256a498b2b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.818746 dv-cinder-master-3-smb devstack@keystone.service[80376]: WARNING py.warnings [None req-7adaba13-d844-453c-8a91-3256a498b2b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:12:18.818746 dv-cinder-master-3-smb devstack@keystone.service[80376]: warnings.warn(msg) Jul 05 10:12:18.818746 dv-cinder-master-3-smb devstack@keystone.service[80376]: Jul 05 10:12:18.819927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaba13-d844-453c-8a91-3256a498b2b9 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.831545 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 774/1570] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:18.832429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f554e3-01e7-4c28-a919-9b827f1a221b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.842526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f554e3-01e7-4c28-a919-9b827f1a221b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.847435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71af67f-ae96-4375-949b-c1010e1c1dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.848228 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71af67f-ae96-4375-949b-c1010e1c1dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.848518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71af67f-ae96-4375-949b-c1010e1c1dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.848796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71af67f-ae96-4375-949b-c1010e1c1dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.849558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71af67f-ae96-4375-949b-c1010e1c1dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.851883 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71af67f-ae96-4375-949b-c1010e1c1dd2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:18.860575 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 797/1571] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:18.860575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaba13-d844-453c-8a91-3256a498b2b9 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.883017 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 775/1572] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:18.886808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71af67f-ae96-4375-949b-c1010e1c1dd2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:18.940893 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaba13-d844-453c-8a91-3256a498b2b9 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.962365 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71af67f-ae96-4375-949b-c1010e1c1dd2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:18.969090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ea4a6-c338-4764-ad69-862880cadf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:18.970152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ea4a6-c338-4764-ad69-862880cadf12 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:18.970685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ea4a6-c338-4764-ad69-862880cadf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:18.971176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ea4a6-c338-4764-ad69-862880cadf12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:18.972232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ea4a6-c338-4764-ad69-862880cadf12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:18.974344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ea4a6-c338-4764-ad69-862880cadf12 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.013341 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 776/1573] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/projects => generated 393 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.014609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ea4a6-c338-4764-ad69-862880cadf12 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.024796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e15f79-32c8-4d80-a0df-215e6a9e011a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.026171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e15f79-32c8-4d80-a0df-215e6a9e011a admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.026171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e15f79-32c8-4d80-a0df-215e6a9e011a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.026795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e15f79-32c8-4d80-a0df-215e6a9e011a admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.027435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e15f79-32c8-4d80-a0df-215e6a9e011a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.029793 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e15f79-32c8-4d80-a0df-215e6a9e011a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.058429 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 798/1574] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:18 2022] POST /identity/v3/projects => generated 371 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.060079 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e15f79-32c8-4d80-a0df-215e6a9e011a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.084261 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ea4a6-c338-4764-ad69-862880cadf12 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.098082 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:19.115369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:19.133426 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e15f79-32c8-4d80-a0df-215e6a9e011a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.143686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92b54d-d8c8-4fdb-bebe-af7c420a4c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.144371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92b54d-d8c8-4fdb-bebe-af7c420a4c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.144653 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92b54d-d8c8-4fdb-bebe-af7c420a4c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.144928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92b54d-d8c8-4fdb-bebe-af7c420a4c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.145627 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92b54d-d8c8-4fdb-bebe-af7c420a4c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.147602 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92b54d-d8c8-4fdb-bebe-af7c420a4c49 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.158943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9440b-e1bd-4549-854b-caa524530a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.159457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9440b-e1bd-4549-854b-caa524530a08 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.159825 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9440b-e1bd-4549-854b-caa524530a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.160025 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9440b-e1bd-4549-854b-caa524530a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.160751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9440b-e1bd-4549-854b-caa524530a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.164206 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9440b-e1bd-4549-854b-caa524530a08 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.176616 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 799/1575] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/projects => generated 389 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.177688 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92b54d-d8c8-4fdb-bebe-af7c420a4c49 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.201776 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 777/1576] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/projects => generated 377 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.202809 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9440b-e1bd-4549-854b-caa524530a08 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.258737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92b54d-d8c8-4fdb-bebe-af7c420a4c49 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.301968 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9440b-e1bd-4549-854b-caa524530a08 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.306907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd47141-61b1-4c22-b3f6-3efee08a7888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.307618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd47141-61b1-4c22-b3f6-3efee08a7888 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.307897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd47141-61b1-4c22-b3f6-3efee08a7888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.308182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd47141-61b1-4c22-b3f6-3efee08a7888 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.309533 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd47141-61b1-4c22-b3f6-3efee08a7888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.310635 dv-cinder-master-3-smb devstack@keystone.service[80377]: WARNING py.warnings [None req-2dd47141-61b1-4c22-b3f6-3efee08a7888 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:12:19.310635 dv-cinder-master-3-smb devstack@keystone.service[80377]: warnings.warn(msg) Jul 05 10:12:19.310635 dv-cinder-master-3-smb devstack@keystone.service[80377]: Jul 05 10:12:19.312017 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd47141-61b1-4c22-b3f6-3efee08a7888 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.322942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f266d9-4787-4653-a674-34366d59bef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.324122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f266d9-4787-4653-a674-34366d59bef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.324671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f266d9-4787-4653-a674-34366d59bef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.325182 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f266d9-4787-4653-a674-34366d59bef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.325932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f266d9-4787-4653-a674-34366d59bef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.328316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f266d9-4787-4653-a674-34366d59bef3 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.339185 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 778/1577] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.343192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd47141-61b1-4c22-b3f6-3efee08a7888 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.355943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd47141-61b1-4c22-b3f6-3efee08a7888 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.361132 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 800/1578] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.361132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f266d9-4787-4653-a674-34366d59bef3 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.411753 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:19.423127 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cbd6d-82b0-4bfb-989f-d3d64c46c25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.423926 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cbd6d-82b0-4bfb-989f-d3d64c46c25c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.424208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cbd6d-82b0-4bfb-989f-d3d64c46c25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.424597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cbd6d-82b0-4bfb-989f-d3d64c46c25c admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.425304 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43cbd6d-82b0-4bfb-989f-d3d64c46c25c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.427274 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43cbd6d-82b0-4bfb-989f-d3d64c46c25c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.428132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f266d9-4787-4653-a674-34366d59bef3 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.433254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae1edf-ade7-4ed2-afd8-65906eec77e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.433946 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae1edf-ade7-4ed2-afd8-65906eec77e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.434247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae1edf-ade7-4ed2-afd8-65906eec77e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.434537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae1edf-ade7-4ed2-afd8-65906eec77e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.435242 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae1edf-ade7-4ed2-afd8-65906eec77e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.437297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae1edf-ade7-4ed2-afd8-65906eec77e3 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.454133 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 779/1579] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.455495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cbd6d-82b0-4bfb-989f-d3d64c46c25c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.463635 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 801/1580] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.464314 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae1edf-ade7-4ed2-afd8-65906eec77e3 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.467372 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cbd6d-82b0-4bfb-989f-d3d64c46c25c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.472221 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e998d8a-6aa4-47d0-877e-bc7002afe62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.472932 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e998d8a-6aa4-47d0-877e-bc7002afe62c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.473222 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e998d8a-6aa4-47d0-877e-bc7002afe62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.473503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e998d8a-6aa4-47d0-877e-bc7002afe62c admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.474206 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e998d8a-6aa4-47d0-877e-bc7002afe62c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.474389 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae1edf-ade7-4ed2-afd8-65906eec77e3 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.476207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e998d8a-6aa4-47d0-877e-bc7002afe62c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.479390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc2ed4-840d-46d4-8b0b-2af409ac2a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.480172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc2ed4-840d-46d4-8b0b-2af409ac2a9a admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.480460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc2ed4-840d-46d4-8b0b-2af409ac2a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.480740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc2ed4-840d-46d4-8b0b-2af409ac2a9a admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.481433 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc2ed4-840d-46d4-8b0b-2af409ac2a9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.483788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc2ed4-840d-46d4-8b0b-2af409ac2a9a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.497998 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 780/1581] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.498867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e998d8a-6aa4-47d0-877e-bc7002afe62c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.505946 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 802/1582] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.507245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853dacb-0984-4add-8f1c-0ec1860d6571 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.507538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853dacb-0984-4add-8f1c-0ec1860d6571 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.507818 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853dacb-0984-4add-8f1c-0ec1860d6571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.512557 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e998d8a-6aa4-47d0-877e-bc7002afe62c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.517285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4df0-2f3e-4596-8071-6432cbcf1fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.517952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4df0-2f3e-4596-8071-6432cbcf1fcd admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.518249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4df0-2f3e-4596-8071-6432cbcf1fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.518548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4df0-2f3e-4596-8071-6432cbcf1fcd admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.519244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b4df0-2f3e-4596-8071-6432cbcf1fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.521201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b4df0-2f3e-4596-8071-6432cbcf1fcd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.534240 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-6853dacb-0984-4add-8f1c-0ec1860d6571 None None] MFA Rules not processed for user `a98ac3936d784bed9aa5b510100deb14`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:19.535479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6853dacb-0984-4add-8f1c-0ec1860d6571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:19.537207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:19.543690 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 781/1583] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.544944 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4df0-2f3e-4596-8071-6432cbcf1fcd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.554626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4df0-2f3e-4596-8071-6432cbcf1fcd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.560256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb35fd0-f859-41a7-b9ae-041f7a4c9df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.560941 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb35fd0-f859-41a7-b9ae-041f7a4c9df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.561223 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb35fd0-f859-41a7-b9ae-041f7a4c9df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.561531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb35fd0-f859-41a7-b9ae-041f7a4c9df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.562252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb35fd0-f859-41a7-b9ae-041f7a4c9df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.563900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb35fd0-f859-41a7-b9ae-041f7a4c9df7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.574669 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 782/1584] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.575415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb35fd0-f859-41a7-b9ae-041f7a4c9df7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.585194 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb35fd0-f859-41a7-b9ae-041f7a4c9df7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.590520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c987080-c3c0-4c56-a4d5-a4628f8ba946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.591195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c987080-c3c0-4c56-a4d5-a4628f8ba946 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.591476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c987080-c3c0-4c56-a4d5-a4628f8ba946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.591749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c987080-c3c0-4c56-a4d5-a4628f8ba946 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.592433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c987080-c3c0-4c56-a4d5-a4628f8ba946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.594493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c987080-c3c0-4c56-a4d5-a4628f8ba946 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.603937 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 803/1585] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:19.604935 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853dacb-0984-4add-8f1c-0ec1860d6571 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.616014 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853dacb-0984-4add-8f1c-0ec1860d6571 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.619121 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 783/1586] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.620075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c987080-c3c0-4c56-a4d5-a4628f8ba946 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.623690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5de0c-8b7a-4e5d-8255-92cf3c2fb9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.624375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5de0c-8b7a-4e5d-8255-92cf3c2fb9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.624683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5de0c-8b7a-4e5d-8255-92cf3c2fb9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.624941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5de0c-8b7a-4e5d-8255-92cf3c2fb9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.625679 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5de0c-8b7a-4e5d-8255-92cf3c2fb9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.627340 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5de0c-8b7a-4e5d-8255-92cf3c2fb9f9 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.629615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c987080-c3c0-4c56-a4d5-a4628f8ba946 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.633991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ae5d-255f-4e76-a258-2c9c75dfb9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.634668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ae5d-255f-4e76-a258-2c9c75dfb9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.634949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ae5d-255f-4e76-a258-2c9c75dfb9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.635222 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ae5d-255f-4e76-a258-2c9c75dfb9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.635945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06ae5d-255f-4e76-a258-2c9c75dfb9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.637355 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 804/1587] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.637544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06ae5d-255f-4e76-a258-2c9c75dfb9e0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.638135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5de0c-8b7a-4e5d-8255-92cf3c2fb9f9 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.648777 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 784/1588] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.649630 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ae5d-255f-4e76-a258-2c9c75dfb9e0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.650730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5de0c-8b7a-4e5d-8255-92cf3c2fb9f9 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.655602 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff6774-3c32-4d95-9819-8ba5aceb0afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.656270 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6774-3c32-4d95-9819-8ba5aceb0afa admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.656553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6774-3c32-4d95-9819-8ba5aceb0afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.656827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6774-3c32-4d95-9819-8ba5aceb0afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.657544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6774-3c32-4d95-9819-8ba5aceb0afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.659225 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6774-3c32-4d95-9819-8ba5aceb0afa admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.663028 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ae5d-255f-4e76-a258-2c9c75dfb9e0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.667760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a62de8-d201-46f4-8694-b1a0b7be88d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.668434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a62de8-d201-46f4-8694-b1a0b7be88d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.668722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a62de8-d201-46f4-8694-b1a0b7be88d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.668999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a62de8-d201-46f4-8694-b1a0b7be88d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.672417 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 805/1589] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.672574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a62de8-d201-46f4-8694-b1a0b7be88d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.672677 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff6774-3c32-4d95-9819-8ba5aceb0afa admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.672769 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a62de8-d201-46f4-8694-b1a0b7be88d5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.682172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff6774-3c32-4d95-9819-8ba5aceb0afa admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.685898 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 785/1590] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.685898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a62de8-d201-46f4-8694-b1a0b7be88d5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.689234 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de92976-c529-408b-8f50-9e9af679886a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.689906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de92976-c529-408b-8f50-9e9af679886a admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.690214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de92976-c529-408b-8f50-9e9af679886a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.690526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de92976-c529-408b-8f50-9e9af679886a admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.691259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de92976-c529-408b-8f50-9e9af679886a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.692993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de92976-c529-408b-8f50-9e9af679886a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.694366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a62de8-d201-46f4-8694-b1a0b7be88d5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.699303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666306e-3798-46bf-b22c-9c0e6a912c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.699981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666306e-3798-46bf-b22c-9c0e6a912c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.700407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666306e-3798-46bf-b22c-9c0e6a912c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.700689 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666306e-3798-46bf-b22c-9c0e6a912c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.701557 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666306e-3798-46bf-b22c-9c0e6a912c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.703202 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666306e-3798-46bf-b22c-9c0e6a912c07 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.703613 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 806/1591] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.704511 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de92976-c529-408b-8f50-9e9af679886a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.715730 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 786/1592] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.719141 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de92976-c529-408b-8f50-9e9af679886a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.719234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666306e-3798-46bf-b22c-9c0e6a912c07 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.722106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9bcea-4115-42fc-8c61-c476d8e7ed57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.722862 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9bcea-4115-42fc-8c61-c476d8e7ed57 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.723181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9bcea-4115-42fc-8c61-c476d8e7ed57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.723435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9bcea-4115-42fc-8c61-c476d8e7ed57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.724153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9bcea-4115-42fc-8c61-c476d8e7ed57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.726115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9bcea-4115-42fc-8c61-c476d8e7ed57 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.728062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666306e-3798-46bf-b22c-9c0e6a912c07 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.732752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805c326-e0d2-443c-bd4e-ac72fe28c061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.733408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805c326-e0d2-443c-bd4e-ac72fe28c061 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.733688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805c326-e0d2-443c-bd4e-ac72fe28c061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.733960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805c326-e0d2-443c-bd4e-ac72fe28c061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.734770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805c326-e0d2-443c-bd4e-ac72fe28c061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.736361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805c326-e0d2-443c-bd4e-ac72fe28c061 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.746957 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 787/1593] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.747855 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805c326-e0d2-443c-bd4e-ac72fe28c061 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.757084 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 807/1594] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:19.757084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9bcea-4115-42fc-8c61-c476d8e7ed57 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.759108 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805c326-e0d2-443c-bd4e-ac72fe28c061 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.817462 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7bd07e-80de-47e8-a32c-825f92f4d0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.818068 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:19.818205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bd07e-80de-47e8-a32c-825f92f4d0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.818492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bd07e-80de-47e8-a32c-825f92f4d0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.818768 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bd07e-80de-47e8-a32c-825f92f4d0b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.819661 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bd07e-80de-47e8-a32c-825f92f4d0b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.821413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bd07e-80de-47e8-a32c-825f92f4d0b6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.824370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9bcea-4115-42fc-8c61-c476d8e7ed57 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.829463 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8feec5-7323-427d-8d1a-e6ac9a41c5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.830135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8feec5-7323-427d-8d1a-e6ac9a41c5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.830461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8feec5-7323-427d-8d1a-e6ac9a41c5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.830739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8feec5-7323-427d-8d1a-e6ac9a41c5d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.834897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8feec5-7323-427d-8d1a-e6ac9a41c5d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.835033 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 788/1595] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.835033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7bd07e-80de-47e8-a32c-825f92f4d0b6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.835224 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8feec5-7323-427d-8d1a-e6ac9a41c5d6 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.845379 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 808/1596] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.846076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8feec5-7323-427d-8d1a-e6ac9a41c5d6 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.846786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7bd07e-80de-47e8-a32c-825f92f4d0b6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.851278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cc8e7-afee-4ab3-bca6-a205ba3a2b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.851951 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cc8e7-afee-4ab3-bca6-a205ba3a2b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.852210 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cc8e7-afee-4ab3-bca6-a205ba3a2b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.852484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cc8e7-afee-4ab3-bca6-a205ba3a2b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.853200 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cc8e7-afee-4ab3-bca6-a205ba3a2b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.854848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cc8e7-afee-4ab3-bca6-a205ba3a2b90 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.855888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8feec5-7323-427d-8d1a-e6ac9a41c5d6 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.860383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2c1d4-69a6-4101-94a0-71feb94fe904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.861029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c1d4-69a6-4101-94a0-71feb94fe904 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.861314 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c1d4-69a6-4101-94a0-71feb94fe904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.861674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c1d4-69a6-4101-94a0-71feb94fe904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.862335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2c1d4-69a6-4101-94a0-71feb94fe904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.863909 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2c1d4-69a6-4101-94a0-71feb94fe904 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.864742 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 789/1597] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.865515 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cc8e7-afee-4ab3-bca6-a205ba3a2b90 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.873926 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 809/1598] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.874726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2c1d4-69a6-4101-94a0-71feb94fe904 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.875384 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cc8e7-afee-4ab3-bca6-a205ba3a2b90 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.881051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d794be-c964-400a-9225-010a955e4320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.881702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d794be-c964-400a-9225-010a955e4320 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.881985 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d794be-c964-400a-9225-010a955e4320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.882292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d794be-c964-400a-9225-010a955e4320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.883043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d794be-c964-400a-9225-010a955e4320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.883986 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2c1d4-69a6-4101-94a0-71feb94fe904 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.884672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d794be-c964-400a-9225-010a955e4320 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.888416 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b13eb-7907-4521-a7e5-b8b32ac4b3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.889307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b13eb-7907-4521-a7e5-b8b32ac4b3fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.889590 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b13eb-7907-4521-a7e5-b8b32ac4b3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.890196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b13eb-7907-4521-a7e5-b8b32ac4b3fd admin admin] PATH_INFO: `/v3/projects/227f2bd5d6d44255ad34e052b505d42c/users/b63fb78fbcfa449db8c0034eb9e12449/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.894834 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 790/1599] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.895677 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d794be-c964-400a-9225-010a955e4320 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.905131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b13eb-7907-4521-a7e5-b8b32ac4b3fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=227f2bd5d6d44255ad34e052b505d42c, user_id=b63fb78fbcfa449db8c0034eb9e12449, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.905482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d794be-c964-400a-9225-010a955e4320 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.906168 dv-cinder-master-3-smb devstack@keystone.service[80376]: WARNING py.warnings [None req-582b13eb-7907-4521-a7e5-b8b32ac4b3fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:12:19.906168 dv-cinder-master-3-smb devstack@keystone.service[80376]: warnings.warn(msg) Jul 05 10:12:19.906168 dv-cinder-master-3-smb devstack@keystone.service[80376]: Jul 05 10:12:19.908184 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b13eb-7907-4521-a7e5-b8b32ac4b3fd admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.910997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb81381-50a6-4d96-acc6-66fbc4339ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.911645 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81381-50a6-4d96-acc6-66fbc4339ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.911928 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81381-50a6-4d96-acc6-66fbc4339ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.912199 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81381-50a6-4d96-acc6-66fbc4339ac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.912914 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb81381-50a6-4d96-acc6-66fbc4339ac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.914980 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb81381-50a6-4d96-acc6-66fbc4339ac7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.922138 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 810/1600] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:19 2022] PUT /identity/v3/projects/227f2bd5d6d44255ad34e052b505d42c/users/b63fb78fbcfa449db8c0034eb9e12449/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:19.922933 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b13eb-7907-4521-a7e5-b8b32ac4b3fd admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.927823 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 791/1601] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:19.929219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb81381-50a6-4d96-acc6-66fbc4339ac7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:19.939669 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb81381-50a6-4d96-acc6-66fbc4339ac7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.985590 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b13eb-7907-4521-a7e5-b8b32ac4b3fd admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:19.986121 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:19.990706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0fdff-fb89-4a7f-8307-4bf71ff9112e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.991488 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0fdff-fb89-4a7f-8307-4bf71ff9112e admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.991772 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0fdff-fb89-4a7f-8307-4bf71ff9112e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.992083 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0fdff-fb89-4a7f-8307-4bf71ff9112e admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:19.992812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0fdff-fb89-4a7f-8307-4bf71ff9112e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:19.994778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0fdff-fb89-4a7f-8307-4bf71ff9112e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:19.997473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094fab7b-e3df-4fd9-bf37-45aa016d7cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:19.998447 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094fab7b-e3df-4fd9-bf37-45aa016d7cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:19.998730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094fab7b-e3df-4fd9-bf37-45aa016d7cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:19.999030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094fab7b-e3df-4fd9-bf37-45aa016d7cfc admin admin] PATH_INFO: `/v3/projects/6209a54f6be649b5ba947a0568f8a777/users/fd1f11d983d84f33aa3f1cd71b2ba813/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.014130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094fab7b-e3df-4fd9-bf37-45aa016d7cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6209a54f6be649b5ba947a0568f8a777, user_id=fd1f11d983d84f33aa3f1cd71b2ba813, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.016213 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 811/1602] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:19 2022] POST /identity/v3/users => generated 308 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:20.016880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094fab7b-e3df-4fd9-bf37-45aa016d7cfc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.016999 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0fdff-fb89-4a7f-8307-4bf71ff9112e admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.026908 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0fdff-fb89-4a7f-8307-4bf71ff9112e admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.031226 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 792/1603] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:19 2022] PUT /identity/v3/projects/6209a54f6be649b5ba947a0568f8a777/users/fd1f11d983d84f33aa3f1cd71b2ba813/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:20.032140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094fab7b-e3df-4fd9-bf37-45aa016d7cfc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.091800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094fab7b-e3df-4fd9-bf37-45aa016d7cfc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.092093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52eba67-97ed-4701-a39b-654a696b8947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.092857 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52eba67-97ed-4701-a39b-654a696b8947 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.093143 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52eba67-97ed-4701-a39b-654a696b8947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.093419 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52eba67-97ed-4701-a39b-654a696b8947 admin admin] PATH_INFO: `/v3/projects/de80b31cd6c144ac838b613648f8f3a7/users/3c6e5faddabd4208b6e476be4b59e834/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.096456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d8677-cd91-42fb-93e2-f482cfb514e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.097343 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d8677-cd91-42fb-93e2-f482cfb514e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.097621 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d8677-cd91-42fb-93e2-f482cfb514e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.097893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d8677-cd91-42fb-93e2-f482cfb514e4 admin admin] PATH_INFO: `/v3/projects/a0b57d468b3a464fb436cb9168c97199/users/adf6b9d16af549a6be1ffdd6cfe0fd6f/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.109470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52eba67-97ed-4701-a39b-654a696b8947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de80b31cd6c144ac838b613648f8f3a7, user_id=3c6e5faddabd4208b6e476be4b59e834, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.112102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d8677-cd91-42fb-93e2-f482cfb514e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b57d468b3a464fb436cb9168c97199, user_id=adf6b9d16af549a6be1ffdd6cfe0fd6f, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.112676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52eba67-97ed-4701-a39b-654a696b8947 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.114751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d8677-cd91-42fb-93e2-f482cfb514e4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.127848 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 812/1604] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:20 2022] PUT /identity/v3/projects/de80b31cd6c144ac838b613648f8f3a7/users/3c6e5faddabd4208b6e476be4b59e834/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:20.128038 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 793/1605] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:20 2022] PUT /identity/v3/projects/a0b57d468b3a464fb436cb9168c97199/users/adf6b9d16af549a6be1ffdd6cfe0fd6f/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:20.128697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52eba67-97ed-4701-a39b-654a696b8947 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.128815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d8677-cd91-42fb-93e2-f482cfb514e4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.167724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:20.191615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d8677-cd91-42fb-93e2-f482cfb514e4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.196327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eab9b9-6358-43f7-9efd-1a8b390238d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.197229 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eab9b9-6358-43f7-9efd-1a8b390238d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.197512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eab9b9-6358-43f7-9efd-1a8b390238d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.197823 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eab9b9-6358-43f7-9efd-1a8b390238d5 admin admin] PATH_INFO: `/v3/projects/cf8cdaba37244ff9bcf6df35373f1467/users/7d4bb910a5bc4c7ea2667dc65066b2bd/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.201479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52eba67-97ed-4701-a39b-654a696b8947 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.206216 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd1c24-f6cd-47a8-ae67-355993d382fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.207180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd1c24-f6cd-47a8-ae67-355993d382fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.207463 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd1c24-f6cd-47a8-ae67-355993d382fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.207740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd1c24-f6cd-47a8-ae67-355993d382fd admin admin] PATH_INFO: `/v3/projects/d343798527cf445295916cc8e8134bf3/users/51e20b9a489342d2b8f0f85542ab370c/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.212712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eab9b9-6358-43f7-9efd-1a8b390238d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf8cdaba37244ff9bcf6df35373f1467, user_id=7d4bb910a5bc4c7ea2667dc65066b2bd, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.215404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eab9b9-6358-43f7-9efd-1a8b390238d5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.223742 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd1c24-f6cd-47a8-ae67-355993d382fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d343798527cf445295916cc8e8134bf3, user_id=51e20b9a489342d2b8f0f85542ab370c, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.226694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd1c24-f6cd-47a8-ae67-355993d382fd admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.229581 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 794/1606] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:20 2022] PUT /identity/v3/projects/cf8cdaba37244ff9bcf6df35373f1467/users/7d4bb910a5bc4c7ea2667dc65066b2bd/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:20.231090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c035030-fe42-4920-8006-be930c00ed59 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.231418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c035030-fe42-4920-8006-be930c00ed59 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.231730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c035030-fe42-4920-8006-be930c00ed59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.245293 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 813/1607] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:20 2022] PUT /identity/v3/projects/d343798527cf445295916cc8e8134bf3/users/51e20b9a489342d2b8f0f85542ab370c/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:20.246102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd1c24-f6cd-47a8-ae67-355993d382fd admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.262196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0c035030-fe42-4920-8006-be930c00ed59 None None] MFA Rules not processed for user `b63fb78fbcfa449db8c0034eb9e12449`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:20.263484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0c035030-fe42-4920-8006-be930c00ed59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:20.292766 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:20.309960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd1c24-f6cd-47a8-ae67-355993d382fd admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.314609 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984dceed-5e29-4239-a11d-80b06cac74f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.315565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dceed-5e29-4239-a11d-80b06cac74f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.315845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dceed-5e29-4239-a11d-80b06cac74f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.316119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dceed-5e29-4239-a11d-80b06cac74f1 admin admin] PATH_INFO: `/v3/projects/e545a9dca63a4f0aa719494f273262af/users/8fc0b54d732e43f38fead2ce492d1641/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.329772 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 795/1608] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:20.330484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c035030-fe42-4920-8006-be930c00ed59 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.331348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984dceed-5e29-4239-a11d-80b06cac74f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e545a9dca63a4f0aa719494f273262af, user_id=8fc0b54d732e43f38fead2ce492d1641, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.334465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984dceed-5e29-4239-a11d-80b06cac74f1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.342535 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c035030-fe42-4920-8006-be930c00ed59 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.349795 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 814/1609] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:20 2022] PUT /identity/v3/projects/e545a9dca63a4f0aa719494f273262af/users/8fc0b54d732e43f38fead2ce492d1641/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:20.351209 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8424e-7c5e-49d0-aecd-0919de786541 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.351510 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8424e-7c5e-49d0-aecd-0919de786541 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.351788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8424e-7c5e-49d0-aecd-0919de786541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.382412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-aab8424e-7c5e-49d0-aecd-0919de786541 None None] MFA Rules not processed for user `fd1f11d983d84f33aa3f1cd71b2ba813`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:20.383669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-aab8424e-7c5e-49d0-aecd-0919de786541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:20.398317 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959f7cd-1855-4ef6-9e6e-a2ea8630a78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.399024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959f7cd-1855-4ef6-9e6e-a2ea8630a78f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.399315 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959f7cd-1855-4ef6-9e6e-a2ea8630a78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.399700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959f7cd-1855-4ef6-9e6e-a2ea8630a78f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.400450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959f7cd-1855-4ef6-9e6e-a2ea8630a78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.402109 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959f7cd-1855-4ef6-9e6e-a2ea8630a78f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.412193 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 796/1610] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:20.413526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77d79a-d414-4ba5-9f9d-fe4440ebdb32 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.413811 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77d79a-d414-4ba5-9f9d-fe4440ebdb32 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.414088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77d79a-d414-4ba5-9f9d-fe4440ebdb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.431384 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7b77d79a-d414-4ba5-9f9d-fe4440ebdb32 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:20.432558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7b77d79a-d414-4ba5-9f9d-fe4440ebdb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:20.440325 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 797/1611] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:20.441629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133970-34d5-45ef-bb70-749bd69d8cde None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.441915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133970-34d5-45ef-bb70-749bd69d8cde None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.442187 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8133970-34d5-45ef-bb70-749bd69d8cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.452397 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 815/1612] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:20.453743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb224-d69f-4d64-af3a-40809aaa1292 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.454035 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb224-d69f-4d64-af3a-40809aaa1292 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.454346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb224-d69f-4d64-af3a-40809aaa1292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.472251 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-c96cb224-d69f-4d64-af3a-40809aaa1292 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:20.472444 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a8133970-34d5-45ef-bb70-749bd69d8cde None None] MFA Rules not processed for user `adf6b9d16af549a6be1ffdd6cfe0fd6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:20.473471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c96cb224-d69f-4d64-af3a-40809aaa1292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:20.473587 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a8133970-34d5-45ef-bb70-749bd69d8cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:20.481753 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 816/1613] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:20.483405 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c3843-f391-4190-ba30-ad62ee489280 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.483690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c3843-f391-4190-ba30-ad62ee489280 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.483962 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c3843-f391-4190-ba30-ad62ee489280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.501865 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-296c3843-f391-4190-ba30-ad62ee489280 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:20.503124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-296c3843-f391-4190-ba30-ad62ee489280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:20.510751 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 817/1614] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:20.511848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c3843-f391-4190-ba30-ad62ee489280 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.526069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:20.545992 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 798/1615] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:20.547698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30948a1c-50e3-4072-bf48-7d1828e591c7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.548216 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30948a1c-50e3-4072-bf48-7d1828e591c7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.548656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30948a1c-50e3-4072-bf48-7d1828e591c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.573558 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c3843-f391-4190-ba30-ad62ee489280 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.575418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-30948a1c-50e3-4072-bf48-7d1828e591c7 None None] MFA Rules not processed for user `8fc0b54d732e43f38fead2ce492d1641`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:20.576740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-30948a1c-50e3-4072-bf48-7d1828e591c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:20.577221 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e2326-3bc1-4999-bfcd-879a9a46b173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.577887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e2326-3bc1-4999-bfcd-879a9a46b173 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.578256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e2326-3bc1-4999-bfcd-879a9a46b173 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.578460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e2326-3bc1-4999-bfcd-879a9a46b173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.616597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:20.634385 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e2326-3bc1-4999-bfcd-879a9a46b173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.635834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e2326-3bc1-4999-bfcd-879a9a46b173 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.642988 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 799/1616] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:20.643788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30948a1c-50e3-4072-bf48-7d1828e591c7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.644835 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 818/1617] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:20.645468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e2326-3bc1-4999-bfcd-879a9a46b173 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.655200 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30948a1c-50e3-4072-bf48-7d1828e591c7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.672304 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce5247-fee2-4d45-88b2-5d8824982bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.679957 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce5247-fee2-4d45-88b2-5d8824982bee admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.680746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce5247-fee2-4d45-88b2-5d8824982bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.681238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce5247-fee2-4d45-88b2-5d8824982bee admin admin] PATH_INFO: `/v3/projects/b0eb3c122aa84bc2a31c2d656d9ecf09/users/406d37c3113745a1a280054b6fc5d2e3/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.703216 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce5247-fee2-4d45-88b2-5d8824982bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0eb3c122aa84bc2a31c2d656d9ecf09, user_id=406d37c3113745a1a280054b6fc5d2e3, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.706799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce5247-fee2-4d45-88b2-5d8824982bee admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.722179 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 800/1618] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:20 2022] PUT /identity/v3/projects/b0eb3c122aa84bc2a31c2d656d9ecf09/users/406d37c3113745a1a280054b6fc5d2e3/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:20.723612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce5247-fee2-4d45-88b2-5d8824982bee admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.730754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e2326-3bc1-4999-bfcd-879a9a46b173 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.792110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f630d-943b-428f-83ed-e402b1ef93f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.793489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f630d-943b-428f-83ed-e402b1ef93f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.793782 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f630d-943b-428f-83ed-e402b1ef93f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.794112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f630d-943b-428f-83ed-e402b1ef93f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.795547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce5247-fee2-4d45-88b2-5d8824982bee admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.800083 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93f0ea-ca58-4a6f-8e38-bd1437e304a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.800791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f0ea-ca58-4a6f-8e38-bd1437e304a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.801135 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f0ea-ca58-4a6f-8e38-bd1437e304a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.801802 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f0ea-ca58-4a6f-8e38-bd1437e304a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.802409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93f0ea-ca58-4a6f-8e38-bd1437e304a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.804197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f630d-943b-428f-83ed-e402b1ef93f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.804374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93f0ea-ca58-4a6f-8e38-bd1437e304a7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.806431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f630d-943b-428f-83ed-e402b1ef93f3 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.816830 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 819/1619] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:20.817557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f630d-943b-428f-83ed-e402b1ef93f3 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.819117 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e49f630d-943b-428f-83ed-e402b1ef93f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:20.831694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f630d-943b-428f-83ed-e402b1ef93f3 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.836005 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9163a-f7c4-4a35-b731-712d41e12e0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.837167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9163a-f7c4-4a35-b731-712d41e12e0e service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.837525 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9163a-f7c4-4a35-b731-712d41e12e0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.837908 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9163a-f7c4-4a35-b731-712d41e12e0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:20.844171 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 801/1620] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:20 2022] POST /identity/v3/projects => generated 405 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:20.844171 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93f0ea-ca58-4a6f-8e38-bd1437e304a7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.850992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9163a-f7c4-4a35-b731-712d41e12e0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:20.853077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9163a-f7c4-4a35-b731-712d41e12e0e service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:20.866144 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 820/1621] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:20.880885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9163a-f7c4-4a35-b731-712d41e12e0e service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:20.904973 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9163a-f7c4-4a35-b731-712d41e12e0e service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.923526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:20.937646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93f0ea-ca58-4a6f-8e38-bd1437e304a7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:20.942620 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c54ad-9977-472a-8377-dc7deaa3d76a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:20.943413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c54ad-9977-472a-8377-dc7deaa3d76a service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:20.943700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c54ad-9977-472a-8377-dc7deaa3d76a service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:20.944359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c54ad-9977-472a-8377-dc7deaa3d76a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.062023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359b05a-4dc7-4a61-aaa7-d1e159a6aecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:21.064130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359b05a-4dc7-4a61-aaa7-d1e159a6aecc admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:21.064573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359b05a-4dc7-4a61-aaa7-d1e159a6aecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:21.065589 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359b05a-4dc7-4a61-aaa7-d1e159a6aecc admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.066594 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359b05a-4dc7-4a61-aaa7-d1e159a6aecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:21.073621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359b05a-4dc7-4a61-aaa7-d1e159a6aecc admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:21.075105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:21.101829 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c54ad-9977-472a-8377-dc7deaa3d76a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:21.106076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c54ad-9977-472a-8377-dc7deaa3d76a service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:21.127130 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 802/1622] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:21.129336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6e71c-25c9-49f5-8ac3-8ea06c3833b7 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:21.129843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6e71c-25c9-49f5-8ac3-8ea06c3833b7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:21.130360 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6e71c-25c9-49f5-8ac3-8ea06c3833b7 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.135570 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 821/1623] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:20 2022] POST /identity/v3/projects => generated 373 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:21.136778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359b05a-4dc7-4a61-aaa7-d1e159a6aecc admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:21.140288 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 803/1624] 10.222.0.16 () {54 vars in 961 bytes} [Tue Jul 5 10:12:21 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:12:21.147627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6e71c-25c9-49f5-8ac3-8ea06c3833b7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:21.251771 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359b05a-4dc7-4a61-aaa7-d1e159a6aecc admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:21.262388 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6e71c-25c9-49f5-8ac3-8ea06c3833b7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:21.272531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9130065-4a82-4e3b-88ce-a989de3c1496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:21.274079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9130065-4a82-4e3b-88ce-a989de3c1496 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:21.274079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9130065-4a82-4e3b-88ce-a989de3c1496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:21.274897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9130065-4a82-4e3b-88ce-a989de3c1496 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.276105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9130065-4a82-4e3b-88ce-a989de3c1496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:21.278505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577ea54-7c82-4bf3-a89c-61874a419770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:21.279711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9130065-4a82-4e3b-88ce-a989de3c1496 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:21.280919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577ea54-7c82-4bf3-a89c-61874a419770 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:21.281288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577ea54-7c82-4bf3-a89c-61874a419770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:21.290263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577ea54-7c82-4bf3-a89c-61874a419770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.291562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577ea54-7c82-4bf3-a89c-61874a419770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:21.293154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577ea54-7c82-4bf3-a89c-61874a419770 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:21.315220 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 822/1625] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:21.316293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577ea54-7c82-4bf3-a89c-61874a419770 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:21.317831 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 804/1626] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:21 2022] POST /identity/v3/users => generated 309 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:21.320379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ff0a5-58aa-4888-bdc2-08ae9c72b878 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:21.320674 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ff0a5-58aa-4888-bdc2-08ae9c72b878 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:21.321144 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ff0a5-58aa-4888-bdc2-08ae9c72b878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.350482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577ea54-7c82-4bf3-a89c-61874a419770 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:21.365143 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bdac9-9cce-434a-ac0a-e1f2f81c6f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:21.366802 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7a1ff0a5-58aa-4888-bdc2-08ae9c72b878 None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:21.367451 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bdac9-9cce-434a-ac0a-e1f2f81c6f3c admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:21.368026 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7a1ff0a5-58aa-4888-bdc2-08ae9c72b878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:21.368844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bdac9-9cce-434a-ac0a-e1f2f81c6f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:21.370393 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bdac9-9cce-434a-ac0a-e1f2f81c6f3c admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.371849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3bdac9-9cce-434a-ac0a-e1f2f81c6f3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:21.375562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3bdac9-9cce-434a-ac0a-e1f2f81c6f3c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:21.423259 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 823/1627] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:21 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:21.431726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bdac9-9cce-434a-ac0a-e1f2f81c6f3c admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:21.459409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bdac9-9cce-434a-ac0a-e1f2f81c6f3c admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:21.475056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a927201-4e7e-4c0d-8808-c93bd4a4ea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:21.476431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a927201-4e7e-4c0d-8808-c93bd4a4ea42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:21.476727 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a927201-4e7e-4c0d-8808-c93bd4a4ea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:21.479696 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a927201-4e7e-4c0d-8808-c93bd4a4ea42 admin admin] PATH_INFO: `/v3/projects/b0eb3c122aa84bc2a31c2d656d9ecf09/users/406d37c3113745a1a280054b6fc5d2e3/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.488334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a927201-4e7e-4c0d-8808-c93bd4a4ea42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0eb3c122aa84bc2a31c2d656d9ecf09, user_id=406d37c3113745a1a280054b6fc5d2e3, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:21.491285 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a927201-4e7e-4c0d-8808-c93bd4a4ea42 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:21.514581 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 805/1628] 10.222.0.16 () {58 vars in 1058 bytes} [Tue Jul 5 10:12:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:21.515819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ff0a5-58aa-4888-bdc2-08ae9c72b878 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:21.527820 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 824/1629] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:21 2022] PUT /identity/v3/projects/b0eb3c122aa84bc2a31c2d656d9ecf09/users/406d37c3113745a1a280054b6fc5d2e3/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:21.532275 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a927201-4e7e-4c0d-8808-c93bd4a4ea42 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:21.534658 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ff0a5-58aa-4888-bdc2-08ae9c72b878 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:21.630444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a927201-4e7e-4c0d-8808-c93bd4a4ea42 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:21.635638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f76c62-272d-47d2-adaa-455db9a53596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:21.636827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f76c62-272d-47d2-adaa-455db9a53596 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:21.636827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f76c62-272d-47d2-adaa-455db9a53596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:21.637045 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bcee3-dbec-4e81-9760-b24eed59b6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:21.637303 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f76c62-272d-47d2-adaa-455db9a53596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.638071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcee3-dbec-4e81-9760-b24eed59b6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:21.638071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcee3-dbec-4e81-9760-b24eed59b6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:21.638322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcee3-dbec-4e81-9760-b24eed59b6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.638570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f76c62-272d-47d2-adaa-455db9a53596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:21.639441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bcee3-dbec-4e81-9760-b24eed59b6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:21.640822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f76c62-272d-47d2-adaa-455db9a53596 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:21.641914 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bcee3-dbec-4e81-9760-b24eed59b6a1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:21.643839 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:21.658539 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 806/1630] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:21.659892 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bcee3-dbec-4e81-9760-b24eed59b6a1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:21.661875 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 825/1631] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:21.663156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f76c62-272d-47d2-adaa-455db9a53596 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:21.672142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:21.677071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f76c62-272d-47d2-adaa-455db9a53596 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:21.685303 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a1bf7-cd38-4e39-b598-2f0917c4546b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:21.686758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a1bf7-cd38-4e39-b598-2f0917c4546b admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:21.687170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a1bf7-cd38-4e39-b598-2f0917c4546b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:21.687538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a1bf7-cd38-4e39-b598-2f0917c4546b admin admin] PATH_INFO: `/v3/projects/abe3db42c8434f9bb254dc338b293417/users/2689d792e7a34ad9bee81d5698252153/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.710246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a1bf7-cd38-4e39-b598-2f0917c4546b admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe3db42c8434f9bb254dc338b293417, user_id=2689d792e7a34ad9bee81d5698252153, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:21.714001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a1bf7-cd38-4e39-b598-2f0917c4546b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:21.738322 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 826/1632] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:21 2022] PUT /identity/v3/projects/abe3db42c8434f9bb254dc338b293417/users/2689d792e7a34ad9bee81d5698252153/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:21.739325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a1bf7-cd38-4e39-b598-2f0917c4546b admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:21.779695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bcee3-dbec-4e81-9760-b24eed59b6a1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:21.856510 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a1bf7-cd38-4e39-b598-2f0917c4546b admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:21.862316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1c5bd-692d-4b70-9e05-97176ca6ca17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:21.863335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1c5bd-692d-4b70-9e05-97176ca6ca17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:21.863624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1c5bd-692d-4b70-9e05-97176ca6ca17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:21.863906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1c5bd-692d-4b70-9e05-97176ca6ca17 admin admin] PATH_INFO: `/v3/projects/6c9fc9bb6162478289310d6b743bfa45/users/4fe0362286d4444b8b644ffced2816e5/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:21.887032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1c5bd-692d-4b70-9e05-97176ca6ca17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c9fc9bb6162478289310d6b743bfa45, user_id=4fe0362286d4444b8b644ffced2816e5, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:21.891047 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1c5bd-692d-4b70-9e05-97176ca6ca17 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:21.911465 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 827/1633] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:21 2022] PUT /identity/v3/projects/6c9fc9bb6162478289310d6b743bfa45/users/4fe0362286d4444b8b644ffced2816e5/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:21.912862 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1c5bd-692d-4b70-9e05-97176ca6ca17 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:21.993557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1c5bd-692d-4b70-9e05-97176ca6ca17 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:22.007288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6611d3-4291-44ae-8224-0b78deab759e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:22.008105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6611d3-4291-44ae-8224-0b78deab759e admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.008467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6611d3-4291-44ae-8224-0b78deab759e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.008811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6611d3-4291-44ae-8224-0b78deab759e admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.009778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6611d3-4291-44ae-8224-0b78deab759e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:22.013321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6611d3-4291-44ae-8224-0b78deab759e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:22.026279 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 828/1634] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:22.026279 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6611d3-4291-44ae-8224-0b78deab759e admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:22.036461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6611d3-4291-44ae-8224-0b78deab759e admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:22.042414 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae09ef8-323e-4500-b80b-dde087eb9ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:22.043439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae09ef8-323e-4500-b80b-dde087eb9ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.043963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae09ef8-323e-4500-b80b-dde087eb9ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.044582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae09ef8-323e-4500-b80b-dde087eb9ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.045714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae09ef8-323e-4500-b80b-dde087eb9ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:22.047623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae09ef8-323e-4500-b80b-dde087eb9ca1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:22.063174 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 829/1635] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:22.064280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae09ef8-323e-4500-b80b-dde087eb9ca1 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:22.075218 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae09ef8-323e-4500-b80b-dde087eb9ca1 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:22.081696 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ece04-4c93-4504-87a6-030d8d683e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:22.082932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ece04-4c93-4504-87a6-030d8d683e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.083447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ece04-4c93-4504-87a6-030d8d683e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.083943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ece04-4c93-4504-87a6-030d8d683e69 admin admin] PATH_INFO: `/v3/projects/abe3db42c8434f9bb254dc338b293417/users/2689d792e7a34ad9bee81d5698252153/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.086521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ece04-4c93-4504-87a6-030d8d683e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe3db42c8434f9bb254dc338b293417, user_id=2689d792e7a34ad9bee81d5698252153, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:22.089988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ece04-4c93-4504-87a6-030d8d683e69 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:22.103706 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 830/1636] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:22 2022] PUT /identity/v3/projects/abe3db42c8434f9bb254dc338b293417/users/2689d792e7a34ad9bee81d5698252153/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:22.104761 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ece04-4c93-4504-87a6-030d8d683e69 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:22.123862 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5daac-13bc-4780-b826-c8a47033bc03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:22.124657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5daac-13bc-4780-b826-c8a47033bc03 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.124943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5daac-13bc-4780-b826-c8a47033bc03 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.125217 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5daac-13bc-4780-b826-c8a47033bc03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.158400 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:22.199749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ece04-4c93-4504-87a6-030d8d683e69 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:22.204984 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610199bc-1f38-4e02-a689-95061972a4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:22.206094 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610199bc-1f38-4e02-a689-95061972a4dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.206766 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610199bc-1f38-4e02-a689-95061972a4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.206912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610199bc-1f38-4e02-a689-95061972a4dc admin admin] PATH_INFO: `/v3/projects/6c9fc9bb6162478289310d6b743bfa45/users/4fe0362286d4444b8b644ffced2816e5/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.209064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610199bc-1f38-4e02-a689-95061972a4dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c9fc9bb6162478289310d6b743bfa45, user_id=4fe0362286d4444b8b644ffced2816e5, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:22.213764 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610199bc-1f38-4e02-a689-95061972a4dc admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:22.219076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5daac-13bc-4780-b826-c8a47033bc03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:22.221268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5daac-13bc-4780-b826-c8a47033bc03 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:22.240644 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 807/1638] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:22.240644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3c450-bcc3-4f37-99e6-48e8d32c10cb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.240644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3c450-bcc3-4f37-99e6-48e8d32c10cb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.240644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3c450-bcc3-4f37-99e6-48e8d32c10cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.242355 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 831/1638] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:22 2022] PUT /identity/v3/projects/6c9fc9bb6162478289310d6b743bfa45/users/4fe0362286d4444b8b644ffced2816e5/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:22.247196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5294629-9dcb-4323-8edd-aaabd6be7ede None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.248648 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5294629-9dcb-4323-8edd-aaabd6be7ede None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.249082 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5294629-9dcb-4323-8edd-aaabd6be7ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.281802 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0af3c450-bcc3-4f37-99e6-48e8d32c10cb None None] MFA Rules not processed for user `2689d792e7a34ad9bee81d5698252153`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:22.283376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0af3c450-bcc3-4f37-99e6-48e8d32c10cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:22.311117 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b5294629-9dcb-4323-8edd-aaabd6be7ede None None] MFA Rules not processed for user `4fe0362286d4444b8b644ffced2816e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:22.312371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b5294629-9dcb-4323-8edd-aaabd6be7ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:22.330458 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:22.434178 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 808/1639] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:22.435615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3c450-bcc3-4f37-99e6-48e8d32c10cb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:22.511009 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 832/1640] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:22 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:22.522090 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5294629-9dcb-4323-8edd-aaabd6be7ede None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:22.568759 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3c450-bcc3-4f37-99e6-48e8d32c10cb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:22.578527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cc11a-ccbd-4add-b3ce-21dd3aa221ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:22.579780 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cc11a-ccbd-4add-b3ce-21dd3aa221ea service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.580067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cc11a-ccbd-4add-b3ce-21dd3aa221ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.580344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cc11a-ccbd-4add-b3ce-21dd3aa221ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.598591 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cc11a-ccbd-4add-b3ce-21dd3aa221ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:22.600446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cc11a-ccbd-4add-b3ce-21dd3aa221ea service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:22.613425 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 809/1641] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:22.615735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff0a97-6ca6-421c-ba80-89467f44bcbc None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.616022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff0a97-6ca6-421c-ba80-89467f44bcbc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.616307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff0a97-6ca6-421c-ba80-89467f44bcbc None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.618650 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 810/1642] 10.222.0.16 () {54 vars in 912 bytes} [Tue Jul 5 10:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:12:22.626524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae79cd-9f15-4d0b-8373-3f0da70e8ba6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.627638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae79cd-9f15-4d0b-8373-3f0da70e8ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.628782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae79cd-9f15-4d0b-8373-3f0da70e8ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.653565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2dae79cd-9f15-4d0b-8373-3f0da70e8ba6 None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:22.655799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2dae79cd-9f15-4d0b-8373-3f0da70e8ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:22.671199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5294629-9dcb-4323-8edd-aaabd6be7ede None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:22.684483 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b469a-e882-4d8c-86d0-8b9cf0b2a387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:22.685271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b469a-e882-4d8c-86d0-8b9cf0b2a387 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.685576 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b469a-e882-4d8c-86d0-8b9cf0b2a387 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.685884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b469a-e882-4d8c-86d0-8b9cf0b2a387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.709675 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b469a-e882-4d8c-86d0-8b9cf0b2a387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:22.711643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b469a-e882-4d8c-86d0-8b9cf0b2a387 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:22.732735 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 833/1643] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:22 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:22.833630 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 811/1644] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:12:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:22.920538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b469a-e882-4d8c-86d0-8b9cf0b2a387 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:22.935099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b469a-e882-4d8c-86d0-8b9cf0b2a387 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:22.947283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d3b31-303f-4c7a-83ef-4d9f9a44ca52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:22.948065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d3b31-303f-4c7a-83ef-4d9f9a44ca52 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:22.948351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d3b31-303f-4c7a-83ef-4d9f9a44ca52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:22.948626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d3b31-303f-4c7a-83ef-4d9f9a44ca52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:22.979841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d3b31-303f-4c7a-83ef-4d9f9a44ca52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:22.981348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d3b31-303f-4c7a-83ef-4d9f9a44ca52 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:23.006711 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 834/1645] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:23.081946 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae79cd-9f15-4d0b-8373-3f0da70e8ba6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:23.096627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae79cd-9f15-4d0b-8373-3f0da70e8ba6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:23.106180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c146f-e5b0-4161-ac81-18dd846e6a57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:23.106894 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c146f-e5b0-4161-ac81-18dd846e6a57 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:23.107350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c146f-e5b0-4161-ac81-18dd846e6a57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:23.107760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c146f-e5b0-4161-ac81-18dd846e6a57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:23.234366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c146f-e5b0-4161-ac81-18dd846e6a57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:23.235859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c146f-e5b0-4161-ac81-18dd846e6a57 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:23.248353 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:23.257460 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 812/1646] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:23.298260 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f6a19-74fe-422c-b4e2-648bca8fc56b None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:23.298726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f6a19-74fe-422c-b4e2-648bca8fc56b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:23.299153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f6a19-74fe-422c-b4e2-648bca8fc56b None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:23.300979 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 835/1647] 10.222.0.16 () {54 vars in 961 bytes} [Tue Jul 5 10:12:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:12:23.327333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5372a-c74d-496a-b9b2-b58c51c0320d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:23.327623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5372a-c74d-496a-b9b2-b58c51c0320d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:23.327905 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5372a-c74d-496a-b9b2-b58c51c0320d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:23.363906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-73a5372a-c74d-496a-b9b2-b58c51c0320d None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:23.365151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-73a5372a-c74d-496a-b9b2-b58c51c0320d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:23.388495 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 813/1648] 10.222.0.16 () {58 vars in 1058 bytes} [Tue Jul 5 10:12:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:23.394820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f133b3-6523-4008-8b01-fae83451b229 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:23.395116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f133b3-6523-4008-8b01-fae83451b229 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:23.395374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f133b3-6523-4008-8b01-fae83451b229 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:23.396992 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 836/1649] 10.222.0.16 () {54 vars in 912 bytes} [Tue Jul 5 10:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:12:23.403424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aed1e-e93d-4e66-bad9-78c63b9456cf None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:23.403716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aed1e-e93d-4e66-bad9-78c63b9456cf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:23.403975 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aed1e-e93d-4e66-bad9-78c63b9456cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:23.456273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-686aed1e-e93d-4e66-bad9-78c63b9456cf None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:23.458108 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-686aed1e-e93d-4e66-bad9-78c63b9456cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:23.487911 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 814/1650] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:12:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:23.592524 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f133b3-6523-4008-8b01-fae83451b229 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:23.611780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f133b3-6523-4008-8b01-fae83451b229 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:23.633109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b2184-c260-4607-89ef-76bbcc4ec2b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:23.636961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b2184-c260-4607-89ef-76bbcc4ec2b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:23.637683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b2184-c260-4607-89ef-76bbcc4ec2b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:23.638826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b2184-c260-4607-89ef-76bbcc4ec2b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:23.675236 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b2184-c260-4607-89ef-76bbcc4ec2b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:23.679645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b2184-c260-4607-89ef-76bbcc4ec2b3 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:23.709267 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 837/1651] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:23.876379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430404d4-e8dd-4ef1-b9d8-64dc92ecaba4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:23.876379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430404d4-e8dd-4ef1-b9d8-64dc92ecaba4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:23.877011 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430404d4-e8dd-4ef1-b9d8-64dc92ecaba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:23.893499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b2184-c260-4607-89ef-76bbcc4ec2b3 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:23.967982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:23.973625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-430404d4-e8dd-4ef1-b9d8-64dc92ecaba4 None None] MFA Rules not processed for user `406d37c3113745a1a280054b6fc5d2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:23.977566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-430404d4-e8dd-4ef1-b9d8-64dc92ecaba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:24.096442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b2184-c260-4607-89ef-76bbcc4ec2b3 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:24.118919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:24.121659 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:24.123180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:24.124439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] PATH_INFO: `/v3/users/fd1f11d983d84f33aa3f1cd71b2ba813` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:24.129314 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd1f11d983d84f33aa3f1cd71b2ba813)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:24.135787 dv-cinder-master-3-smb devstack@keystone.service[80376]: WARNING py.warnings [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:12:24.135787 dv-cinder-master-3-smb devstack@keystone.service[80376]: warnings.warn(msg) Jul 05 10:12:24.135787 dv-cinder-master-3-smb devstack@keystone.service[80376]: Jul 05 10:12:24.142916 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:24.214142 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 815/1652] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:23 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:24.320619 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd1f11d983d84f33aa3f1cd71b2ba813'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:24.321819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd1f11d983d84f33aa3f1cd71b2ba813'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:24.388123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd1f11d983d84f33aa3f1cd71b2ba813'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:24.415721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd1f11d983d84f33aa3f1cd71b2ba813'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:24.417560 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd1f11d983d84f33aa3f1cd71b2ba813'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:24.465638 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 838/1653] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:23 2022] DELETE /identity/v3/users/fd1f11d983d84f33aa3f1cd71b2ba813 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:24.471370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a03988-1b8e-4791-a60d-be9e03ef12ad None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:24.471753 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a03988-1b8e-4791-a60d-be9e03ef12ad None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:24.472050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a03988-1b8e-4791-a60d-be9e03ef12ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:24.518447 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-90a03988-1b8e-4791-a60d-be9e03ef12ad None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:24.524376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-90a03988-1b8e-4791-a60d-be9e03ef12ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:24.652252 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 816/1654] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:24.661548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:24.664462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:24.746147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c22be1-e72e-4189-a931-64721bb2e260 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:24.774049 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5f177-ab1d-4b68-80ee-bf8fc657cedc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:24.775259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5f177-ab1d-4b68-80ee-bf8fc657cedc service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:24.776020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5f177-ab1d-4b68-80ee-bf8fc657cedc service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:24.776569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5f177-ab1d-4b68-80ee-bf8fc657cedc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:24.791098 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5f177-ab1d-4b68-80ee-bf8fc657cedc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:24.793262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5f177-ab1d-4b68-80ee-bf8fc657cedc service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:24.806649 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 839/1655] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:25.818746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d45ca4-3653-47fc-bf38-1d5e05c8aeb5 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:25.818746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d45ca4-3653-47fc-bf38-1d5e05c8aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:25.818746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d45ca4-3653-47fc-bf38-1d5e05c8aeb5 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:25.820353 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 817/1656] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:12:25.875577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9c84e-aa66-43d1-af68-b7f44551421b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:25.876211 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9c84e-aa66-43d1-af68-b7f44551421b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:25.876346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9c84e-aa66-43d1-af68-b7f44551421b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:25.928618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-1df9c84e-aa66-43d1-af68-b7f44551421b None None] MFA Rules not processed for user `e9d4a50f842a48b1875900426c6a1ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:25.932555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1df9c84e-aa66-43d1-af68-b7f44551421b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:26.080162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:26.200637 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 840/1657] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:12:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:12:26.204560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f35ad-4467-4aec-8d6d-d5e8a9ab2844 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:26.204560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f35ad-4467-4aec-8d6d-d5e8a9ab2844 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:26.204560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f35ad-4467-4aec-8d6d-d5e8a9ab2844 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:26.204560 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 818/1658] 10.222.0.16 () {54 vars in 960 bytes} [Tue Jul 5 10:12:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:12:26.249649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe333c5-f145-49c7-a120-2f5ddd1ffb34 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:26.249649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe333c5-f145-49c7-a120-2f5ddd1ffb34 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:26.249649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe333c5-f145-49c7-a120-2f5ddd1ffb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:26.313406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-8fe333c5-f145-49c7-a120-2f5ddd1ffb34 None None] MFA Rules not processed for user `e9d4a50f842a48b1875900426c6a1ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:26.318490 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8fe333c5-f145-49c7-a120-2f5ddd1ffb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:26.366979 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 841/1659] 10.222.0.16 () {58 vars in 1057 bytes} [Tue Jul 5 10:12:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:26.367659 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f35ad-4467-4aec-8d6d-d5e8a9ab2844 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:26.416973 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f35ad-4467-4aec-8d6d-d5e8a9ab2844 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:26.431904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bdc3b-11b8-414b-af86-ec938cfcd209 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:26.433429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bdc3b-11b8-414b-af86-ec938cfcd209 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:26.435605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bdc3b-11b8-414b-af86-ec938cfcd209 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:26.437632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bdc3b-11b8-414b-af86-ec938cfcd209 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:26.467448 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bdc3b-11b8-414b-af86-ec938cfcd209 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:26.472298 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bdc3b-11b8-414b-af86-ec938cfcd209 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:26.512366 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 819/1660] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:12:27.210019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe333c5-f145-49c7-a120-2f5ddd1ffb34 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:27.213630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8fe333c5-f145-49c7-a120-2f5ddd1ffb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:27.258634 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe333c5-f145-49c7-a120-2f5ddd1ffb34 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:27.269749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:27.271168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:27.272687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:27.273697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] PATH_INFO: `/v3/projects/6209a54f6be649b5ba947a0568f8a777` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:27.275338 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6209a54f6be649b5ba947a0568f8a777)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:27.280269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:27.373283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6209a54f6be649b5ba947a0568f8a777', 'request_id': 'req-56f8b641-af4c-43d1-b149-43e7391b0664'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:27.380149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6209a54f6be649b5ba947a0568f8a777', 'request_id': 'req-56f8b641-af4c-43d1-b149-43e7391b0664'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:27.395173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6209a54f6be649b5ba947a0568f8a777', 'request_id': 'req-56f8b641-af4c-43d1-b149-43e7391b0664'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:27.433074 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] The token cache is being invalidate because project 6209a54f6be649b5ba947a0568f8a777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:27.433747 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:27.434686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:27.436918 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 842/1661] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:27 2022] DELETE /identity/v3/projects/6209a54f6be649b5ba947a0568f8a777 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:27.441793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bdc3b-11b8-414b-af86-ec938cfcd209 service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:27.459723 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bdc3b-11b8-414b-af86-ec938cfcd209 service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:27.592696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d1784-3309-4bdc-8f75-8a577dc3a3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:27.593623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d1784-3309-4bdc-8f75-8a577dc3a3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:27.593918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d1784-3309-4bdc-8f75-8a577dc3a3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:27.594192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d1784-3309-4bdc-8f75-8a577dc3a3ea admin admin] PATH_INFO: `/v3/users/4fe0362286d4444b8b644ffced2816e5` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:27.595650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d1784-3309-4bdc-8f75-8a577dc3a3ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fe0362286d4444b8b644ffced2816e5)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:27.597654 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d1784-3309-4bdc-8f75-8a577dc3a3ea admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:27.601726 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:27.721483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec5d1784-3309-4bdc-8f75-8a577dc3a3ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fe0362286d4444b8b644ffced2816e5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:27.723001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec5d1784-3309-4bdc-8f75-8a577dc3a3ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fe0362286d4444b8b644ffced2816e5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:27.724576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec5d1784-3309-4bdc-8f75-8a577dc3a3ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fe0362286d4444b8b644ffced2816e5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:27.734932 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec5d1784-3309-4bdc-8f75-8a577dc3a3ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fe0362286d4444b8b644ffced2816e5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:27.752039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec5d1784-3309-4bdc-8f75-8a577dc3a3ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fe0362286d4444b8b644ffced2816e5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:27.775555 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 820/1662] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:27 2022] DELETE /identity/v3/users/4fe0362286d4444b8b644ffced2816e5 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:12:29.094467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:29.096064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:29.120676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8b641-af4c-43d1-b149-43e7391b0664 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:29.349159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:29.351840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:29.352133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:29.352411 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] PATH_INFO: `/v3/projects/6c9fc9bb6162478289310d6b743bfa45` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:29.356371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c9fc9bb6162478289310d6b743bfa45)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:29.359763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:29.486963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c9fc9bb6162478289310d6b743bfa45', 'request_id': 'req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:29.512303 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c9fc9bb6162478289310d6b743bfa45', 'request_id': 'req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:29.516854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:29.542315 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c9fc9bb6162478289310d6b743bfa45', 'request_id': 'req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:29.579384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] The token cache is being invalidate because project 6c9fc9bb6162478289310d6b743bfa45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:29.580543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:29.584882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c1a43c7f-62a1-407d-8782-9bfc15c7f9ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:29.588839 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 843/1663] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:29 2022] DELETE /identity/v3/projects/6c9fc9bb6162478289310d6b743bfa45 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:29.595388 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f45b7-5607-4546-8867-6c6fea306ddc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:29.595889 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f45b7-5607-4546-8867-6c6fea306ddc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:29.595971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f45b7-5607-4546-8867-6c6fea306ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:29.627893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6b0f45b7-5607-4546-8867-6c6fea306ddc None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:29.632937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6b0f45b7-5607-4546-8867-6c6fea306ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:29.783647 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 821/1664] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:29.786318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcb5db-9ea2-420f-ab5c-8e464dfc6363 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:29.786681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcb5db-9ea2-420f-ab5c-8e464dfc6363 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:29.786870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcb5db-9ea2-420f-ab5c-8e464dfc6363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:29.849889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b2bcb5db-9ea2-420f-ab5c-8e464dfc6363 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:29.857723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b2bcb5db-9ea2-420f-ab5c-8e464dfc6363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:29.889960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016154fd-9ec1-48d9-87cb-5026692b30c0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:29.890518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016154fd-9ec1-48d9-87cb-5026692b30c0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:29.890947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016154fd-9ec1-48d9-87cb-5026692b30c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:29.903630 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 844/1665] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:29.904441 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcb5db-9ea2-420f-ab5c-8e464dfc6363 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:29.926909 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcb5db-9ea2-420f-ab5c-8e464dfc6363 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:29.933192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d68970-86c2-4c46-9970-7508ebc8d716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:29.933597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-016154fd-9ec1-48d9-87cb-5026692b30c0 None None] MFA Rules not processed for user `3c6e5faddabd4208b6e476be4b59e834`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:29.935380 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-016154fd-9ec1-48d9-87cb-5026692b30c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:29.935505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d68970-86c2-4c46-9970-7508ebc8d716 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:29.935771 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d68970-86c2-4c46-9970-7508ebc8d716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:29.936053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d68970-86c2-4c46-9970-7508ebc8d716 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:29.936781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d68970-86c2-4c46-9970-7508ebc8d716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:29.937568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d68970-86c2-4c46-9970-7508ebc8d716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:29.939188 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d68970-86c2-4c46-9970-7508ebc8d716 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:29.972642 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 845/1666] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:29.976415 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d68970-86c2-4c46-9970-7508ebc8d716 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:30.007379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d68970-86c2-4c46-9970-7508ebc8d716 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:30.017791 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acb4f3-9673-4a47-81b5-2e39fd73d494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:30.018514 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acb4f3-9673-4a47-81b5-2e39fd73d494 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:30.018789 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acb4f3-9673-4a47-81b5-2e39fd73d494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:30.019063 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acb4f3-9673-4a47-81b5-2e39fd73d494 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:30.022696 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acb4f3-9673-4a47-81b5-2e39fd73d494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:30.028755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acb4f3-9673-4a47-81b5-2e39fd73d494 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:30.104447 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 846/1667] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:29 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:30.107870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acb4f3-9673-4a47-81b5-2e39fd73d494 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:30.111039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c4acb4f3-9673-4a47-81b5-2e39fd73d494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:30.161861 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 822/1668] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:12:30.163624 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016154fd-9ec1-48d9-87cb-5026692b30c0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:30.207583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016154fd-9ec1-48d9-87cb-5026692b30c0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:30.213220 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:30.315023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f607eb-ce44-4c33-b64d-7e594b2ba521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:30.315865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f607eb-ce44-4c33-b64d-7e594b2ba521 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:30.316861 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f607eb-ce44-4c33-b64d-7e594b2ba521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:30.317163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f607eb-ce44-4c33-b64d-7e594b2ba521 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:30.317921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f607eb-ce44-4c33-b64d-7e594b2ba521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:30.320807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f607eb-ce44-4c33-b64d-7e594b2ba521 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:30.352378 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acb4f3-9673-4a47-81b5-2e39fd73d494 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:30.352735 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 823/1669] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:30 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:30.354308 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cf1bc-625e-4514-be56-fff834b88c92 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:30.354662 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cf1bc-625e-4514-be56-fff834b88c92 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:30.355002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cf1bc-625e-4514-be56-fff834b88c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:30.371672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:30.374085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:30.375243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:30.376492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] PATH_INFO: `/v3/users/a98ac3936d784bed9aa5b510100deb14` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:30.379792 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a98ac3936d784bed9aa5b510100deb14)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:30.382927 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b33cf1bc-625e-4514-be56-fff834b88c92 None None] MFA Rules not processed for user `3c6e5faddabd4208b6e476be4b59e834`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:30.383993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:30.384232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b33cf1bc-625e-4514-be56-fff834b88c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:30.494643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a98ac3936d784bed9aa5b510100deb14'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:30.496533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a98ac3936d784bed9aa5b510100deb14'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:30.515312 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 824/1670] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:30.515998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cf1bc-625e-4514-be56-fff834b88c92 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:30.520733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a98ac3936d784bed9aa5b510100deb14'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:30.521977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b33cf1bc-625e-4514-be56-fff834b88c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:30.540413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a98ac3936d784bed9aa5b510100deb14'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:30.541764 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a98ac3936d784bed9aa5b510100deb14'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:30.545001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cf1bc-625e-4514-be56-fff834b88c92 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:30.555443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:30.557219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:30.557663 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:30.558613 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] PATH_INFO: `/v3/users/8fc0b54d732e43f38fead2ce492d1641` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:30.561076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc0b54d732e43f38fead2ce492d1641)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:30.566953 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:30.568457 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 847/1671] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:30 2022] DELETE /identity/v3/users/a98ac3936d784bed9aa5b510100deb14 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:30.571382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:30.588347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:30.653007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc0b54d732e43f38fead2ce492d1641'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:30.654441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc0b54d732e43f38fead2ce492d1641'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:30.655582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc0b54d732e43f38fead2ce492d1641'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:30.670049 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:30.691076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc0b54d732e43f38fead2ce492d1641'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:30.727009 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc0b54d732e43f38fead2ce492d1641'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:30.746183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe731f-4058-4b80-9f01-16df23afcba1 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:30.766128 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 825/1672] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:30 2022] DELETE /identity/v3/users/8fc0b54d732e43f38fead2ce492d1641 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:30.770299 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecd3a9-64e6-49a8-a3bf-11cbf41a8f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:30.770949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:30.771073 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd3a9-64e6-49a8-a3bf-11cbf41a8f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:30.771271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd3a9-64e6-49a8-a3bf-11cbf41a8f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:30.771684 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd3a9-64e6-49a8-a3bf-11cbf41a8f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:30.772824 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecd3a9-64e6-49a8-a3bf-11cbf41a8f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:30.776361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:30.777353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecd3a9-64e6-49a8-a3bf-11cbf41a8f1e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:30.805207 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 848/1673] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:30.809513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c124e-30ee-4736-9ed8-20ce081b6d5a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:30.809779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c124e-30ee-4736-9ed8-20ce081b6d5a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:30.810061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c124e-30ee-4736-9ed8-20ce081b6d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:30.835716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:30.857535 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-663c124e-30ee-4736-9ed8-20ce081b6d5a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:30.858883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-663c124e-30ee-4736-9ed8-20ce081b6d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:30.954059 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 849/1674] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:30.955271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c124e-30ee-4736-9ed8-20ce081b6d5a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:30.960184 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-663c124e-30ee-4736-9ed8-20ce081b6d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:30.979705 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c1201-c965-47d2-b25b-897f6516118a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:30.989731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819ac7e-8086-4a1f-91a7-c30ab2290386 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:30.999885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819ac7e-8086-4a1f-91a7-c30ab2290386 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:31.002787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c124e-30ee-4736-9ed8-20ce081b6d5a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:31.003791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819ac7e-8086-4a1f-91a7-c30ab2290386 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:31.004261 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819ac7e-8086-4a1f-91a7-c30ab2290386 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:31.009695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7819ac7e-8086-4a1f-91a7-c30ab2290386 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:31.022322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de744ff0-c047-4dbd-b1ba-fb0208054197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:31.023114 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de744ff0-c047-4dbd-b1ba-fb0208054197 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:31.023341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de744ff0-c047-4dbd-b1ba-fb0208054197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:31.023619 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de744ff0-c047-4dbd-b1ba-fb0208054197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:31.024368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de744ff0-c047-4dbd-b1ba-fb0208054197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:31.026037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de744ff0-c047-4dbd-b1ba-fb0208054197 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:31.049983 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 850/1675] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:31.051257 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de744ff0-c047-4dbd-b1ba-fb0208054197 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:31.054717 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-de744ff0-c047-4dbd-b1ba-fb0208054197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:31.119560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819ac7e-8086-4a1f-91a7-c30ab2290386 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:31.124486 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819ac7e-8086-4a1f-91a7-c30ab2290386 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:31.165100 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 826/1676] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:31.166031 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819ac7e-8086-4a1f-91a7-c30ab2290386 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:31.167479 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7819ac7e-8086-4a1f-91a7-c30ab2290386 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:31.181638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819ac7e-8086-4a1f-91a7-c30ab2290386 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:31.188159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:31.189263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:31.189549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:31.189831 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] PATH_INFO: `/v3/projects/91a8c19c62ad4d41aca91d7edcf00591` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:31.204021 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de744ff0-c047-4dbd-b1ba-fb0208054197 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:31.205576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91a8c19c62ad4d41aca91d7edcf00591)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:31.206826 dv-cinder-master-3-smb devstack@keystone.service[80377]: WARNING py.warnings [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:12:31.206826 dv-cinder-master-3-smb devstack@keystone.service[80377]: warnings.warn(msg) Jul 05 10:12:31.206826 dv-cinder-master-3-smb devstack@keystone.service[80377]: Jul 05 10:12:31.208572 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:31.214690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8149e47-42c9-4575-8c46-0f737b4fab71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:31.215385 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8149e47-42c9-4575-8c46-0f737b4fab71 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:31.215674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8149e47-42c9-4575-8c46-0f737b4fab71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:31.215952 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8149e47-42c9-4575-8c46-0f737b4fab71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:31.238210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8149e47-42c9-4575-8c46-0f737b4fab71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:31.242128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8149e47-42c9-4575-8c46-0f737b4fab71 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:31.257034 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 851/1677] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:31.258676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8149e47-42c9-4575-8c46-0f737b4fab71 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:31.313598 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8149e47-42c9-4575-8c46-0f737b4fab71 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:31.321806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b71907-5965-45ec-b529-cbb03dd1ff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:31.323245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b71907-5965-45ec-b529-cbb03dd1ff4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:31.323687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b71907-5965-45ec-b529-cbb03dd1ff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:31.324109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b71907-5965-45ec-b529-cbb03dd1ff4e admin admin] PATH_INFO: `/v3/projects/d8085f77d77f41249c68369b093d68cc/users/33370bcfbc8c4bf4ac33a62897d9f513/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:31.348487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b71907-5965-45ec-b529-cbb03dd1ff4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8085f77d77f41249c68369b093d68cc, user_id=33370bcfbc8c4bf4ac33a62897d9f513, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:31.352412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b71907-5965-45ec-b529-cbb03dd1ff4e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:31.400387 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91a8c19c62ad4d41aca91d7edcf00591', 'request_id': 'req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:31.423723 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91a8c19c62ad4d41aca91d7edcf00591', 'request_id': 'req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:31.436736 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 852/1678] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:31 2022] PUT /identity/v3/projects/d8085f77d77f41249c68369b093d68cc/users/33370bcfbc8c4bf4ac33a62897d9f513/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:31.438582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e21e7-9fbe-4808-8c17-e0d88318f3fa None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:31.438877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e21e7-9fbe-4808-8c17-e0d88318f3fa None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:31.439153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e21e7-9fbe-4808-8c17-e0d88318f3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:31.446085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91a8c19c62ad4d41aca91d7edcf00591', 'request_id': 'req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:31.514917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-8c7e21e7-9fbe-4808-8c17-e0d88318f3fa None None] MFA Rules not processed for user `33370bcfbc8c4bf4ac33a62897d9f513`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:31.516172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8c7e21e7-9fbe-4808-8c17-e0d88318f3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:31.527835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] The token cache is being invalidate because project 91a8c19c62ad4d41aca91d7edcf00591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:31.528029 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:31.528889 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:31.534116 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 827/1679] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:31 2022] DELETE /identity/v3/projects/91a8c19c62ad4d41aca91d7edcf00591 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:31.535179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:31.540570 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:31.577641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:31.660823 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:31.720116 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 853/1680] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:31.720776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e21e7-9fbe-4808-8c17-e0d88318f3fa None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:31.723771 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8c7e21e7-9fbe-4808-8c17-e0d88318f3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:31.726120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995160d0-c4c8-47ee-8a5f-09e8ed146ac1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:31.903519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e21e7-9fbe-4808-8c17-e0d88318f3fa None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:31.913987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:31.914938 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:31.915190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:31.915472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] PATH_INFO: `/v3/users/406d37c3113745a1a280054b6fc5d2e3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:31.921882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3c9e7-172b-4033-8cd3-ac88330f3d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:31.921882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3c9e7-172b-4033-8cd3-ac88330f3d70 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:31.921882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3c9e7-172b-4033-8cd3-ac88330f3d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:31.922972 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3c9e7-172b-4033-8cd3-ac88330f3d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:31.922972 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3c9e7-172b-4033-8cd3-ac88330f3d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:31.928839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] RBAC: Authorizing `identity:delete_user(user_id=406d37c3113745a1a280054b6fc5d2e3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:31.931457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:31.931592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3c9e7-172b-4033-8cd3-ac88330f3d70 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:32.026963 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 854/1681] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:31 2022] POST /identity/v3/projects => generated 379 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:32.031637 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3c9e7-172b-4033-8cd3-ac88330f3d70 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:32.072551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '406d37c3113745a1a280054b6fc5d2e3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:32.073795 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '406d37c3113745a1a280054b6fc5d2e3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:32.075228 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '406d37c3113745a1a280054b6fc5d2e3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:32.101132 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '406d37c3113745a1a280054b6fc5d2e3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:32.135703 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '406d37c3113745a1a280054b6fc5d2e3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:32.158813 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 828/1682] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:31 2022] DELETE /identity/v3/users/406d37c3113745a1a280054b6fc5d2e3 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:32.190629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3c9e7-172b-4033-8cd3-ac88330f3d70 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:32.279175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:32.301150 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd246d15-4ca1-4a02-87a7-5847501db88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:32.303455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd246d15-4ca1-4a02-87a7-5847501db88a admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:32.304168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd246d15-4ca1-4a02-87a7-5847501db88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:32.304734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd246d15-4ca1-4a02-87a7-5847501db88a admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:32.306734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd246d15-4ca1-4a02-87a7-5847501db88a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:32.309214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd246d15-4ca1-4a02-87a7-5847501db88a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:32.378819 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 855/1683] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:32 2022] POST /identity/v3/users => generated 296 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:32.382657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:32.384466 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:32.410694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2445ce-090e-447f-947a-86864904bdea admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:32.425300 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c9f1b-0aef-4357-b910-5a847ee86ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:32.426015 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c9f1b-0aef-4357-b910-5a847ee86ede admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:32.426328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c9f1b-0aef-4357-b910-5a847ee86ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:32.426616 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c9f1b-0aef-4357-b910-5a847ee86ede admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:32.427372 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c9f1b-0aef-4357-b910-5a847ee86ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:32.429413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c9f1b-0aef-4357-b910-5a847ee86ede admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:32.458477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd246d15-4ca1-4a02-87a7-5847501db88a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:32.458710 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 829/1684] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:32.480898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd246d15-4ca1-4a02-87a7-5847501db88a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:32.493033 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1d483-de03-4776-bdf4-90f8b0a9e4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:32.493999 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1d483-de03-4776-bdf4-90f8b0a9e4d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:32.494325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1d483-de03-4776-bdf4-90f8b0a9e4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:32.494606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1d483-de03-4776-bdf4-90f8b0a9e4d8 admin admin] PATH_INFO: `/v3/projects/afeb30b15e134306a3dafd0f00f63c9e/users/70178178a21e43faaf3170fda862ca34/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:32.543982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1d483-de03-4776-bdf4-90f8b0a9e4d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afeb30b15e134306a3dafd0f00f63c9e, user_id=70178178a21e43faaf3170fda862ca34, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:32.546270 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1d483-de03-4776-bdf4-90f8b0a9e4d8 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:32.581052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c9f1b-0aef-4357-b910-5a847ee86ede admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:32.581521 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 856/1685] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:32 2022] PUT /identity/v3/projects/afeb30b15e134306a3dafd0f00f63c9e/users/70178178a21e43faaf3170fda862ca34/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:32.599865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c9f1b-0aef-4357-b910-5a847ee86ede admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:32.700097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1d483-de03-4776-bdf4-90f8b0a9e4d8 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:32.701937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d1a1d483-de03-4776-bdf4-90f8b0a9e4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:32.712535 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:32.744601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ead2e-dda4-4add-ad33-e38a2c572daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:32.745409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ead2e-dda4-4add-ad33-e38a2c572daa admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:32.745772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ead2e-dda4-4add-ad33-e38a2c572daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:32.747504 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ead2e-dda4-4add-ad33-e38a2c572daa admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:32.748420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ead2e-dda4-4add-ad33-e38a2c572daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:32.753470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ead2e-dda4-4add-ad33-e38a2c572daa admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:32.778707 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 830/1686] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:12:32.782927 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ead2e-dda4-4add-ad33-e38a2c572daa admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:32.842213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ead2e-dda4-4add-ad33-e38a2c572daa admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:32.863039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657dbf2-661a-4048-9a76-04f025bdc5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:32.864201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657dbf2-661a-4048-9a76-04f025bdc5d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:32.864484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657dbf2-661a-4048-9a76-04f025bdc5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:32.864760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657dbf2-661a-4048-9a76-04f025bdc5d0 admin admin] PATH_INFO: `/v3/projects/afeb30b15e134306a3dafd0f00f63c9e/users/70178178a21e43faaf3170fda862ca34/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:32.873441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657dbf2-661a-4048-9a76-04f025bdc5d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afeb30b15e134306a3dafd0f00f63c9e, user_id=70178178a21e43faaf3170fda862ca34, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:32.883948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1d483-de03-4776-bdf4-90f8b0a9e4d8 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:32.889551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657dbf2-661a-4048-9a76-04f025bdc5d0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:32.894053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:32.895313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:32.895797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:32.896269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] PATH_INFO: `/v3/projects/b0eb3c122aa84bc2a31c2d656d9ecf09` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:32.898639 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0eb3c122aa84bc2a31c2d656d9ecf09)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:32.901203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:32.937677 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 831/1687] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:32 2022] PUT /identity/v3/projects/afeb30b15e134306a3dafd0f00f63c9e/users/70178178a21e43faaf3170fda862ca34/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:12:32.943866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdb09a-91ab-4cbb-b050-9fa57f02fb18 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:32.946100 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdb09a-91ab-4cbb-b050-9fa57f02fb18 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:32.946467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdb09a-91ab-4cbb-b050-9fa57f02fb18 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:32.953299 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 832/1688] 10.222.0.16 () {54 vars in 912 bytes} [Tue Jul 5 10:12:32 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:12:32.957146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ab761-135a-4628-bb42-621011f0f727 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:32.961701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ab761-135a-4628-bb42-621011f0f727 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:32.962073 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ab761-135a-4628-bb42-621011f0f727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:33.036705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0eb3c122aa84bc2a31c2d656d9ecf09', 'request_id': 'req-e67f5b59-dc07-4efe-a685-cf85215224c1'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:33.041764 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c45ab761-135a-4628-bb42-621011f0f727 None None] MFA Rules not processed for user `70178178a21e43faaf3170fda862ca34`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:33.046213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c45ab761-135a-4628-bb42-621011f0f727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:33.055067 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0eb3c122aa84bc2a31c2d656d9ecf09', 'request_id': 'req-e67f5b59-dc07-4efe-a685-cf85215224c1'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:33.070286 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0eb3c122aa84bc2a31c2d656d9ecf09', 'request_id': 'req-e67f5b59-dc07-4efe-a685-cf85215224c1'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:33.104000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] The token cache is being invalidate because project b0eb3c122aa84bc2a31c2d656d9ecf09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:33.104417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:33.106700 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e67f5b59-dc07-4efe-a685-cf85215224c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:33.109080 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 857/1689] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:32 2022] DELETE /identity/v3/projects/b0eb3c122aa84bc2a31c2d656d9ecf09 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:33.110575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8ae56-4e9c-4d26-9f7b-9fd3b0106c3a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:33.110988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8ae56-4e9c-4d26-9f7b-9fd3b0106c3a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:33.111887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8ae56-4e9c-4d26-9f7b-9fd3b0106c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:33.179927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ebd8ae56-4e9c-4d26-9f7b-9fd3b0106c3a None None] MFA Rules not processed for user `e0865f21da0a4c95a84c0099a1b9a78b`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:33.183817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ebd8ae56-4e9c-4d26-9f7b-9fd3b0106c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:33.256662 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 833/1690] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:33.259543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adb338-31cc-49db-b4ec-080b404bdb9d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:33.260538 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adb338-31cc-49db-b4ec-080b404bdb9d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:33.261999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adb338-31cc-49db-b4ec-080b404bdb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:33.277573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:33.317977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c1adb338-31cc-49db-b4ec-080b404bdb9d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:33.323967 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c1adb338-31cc-49db-b4ec-080b404bdb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:33.344852 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 858/1691] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:12:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:33.351531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8ae56-4e9c-4d26-9f7b-9fd3b0106c3a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:33.364683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ebd8ae56-4e9c-4d26-9f7b-9fd3b0106c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:33.467096 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 834/1692] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:33.468384 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adb338-31cc-49db-b4ec-080b404bdb9d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:33.471690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c1adb338-31cc-49db-b4ec-080b404bdb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:33.500560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adb338-31cc-49db-b4ec-080b404bdb9d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:33.510532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e5bed-8345-4000-8519-685620f98347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:33.512327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e5bed-8345-4000-8519-685620f98347 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:33.513232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e5bed-8345-4000-8519-685620f98347 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:33.514376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e5bed-8345-4000-8519-685620f98347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:33.524267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8ae56-4e9c-4d26-9f7b-9fd3b0106c3a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:33.536555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a7d2-cac1-4ba5-a487-a08997b19367 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:33.536742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e5bed-8345-4000-8519-685620f98347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:33.536835 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a7d2-cac1-4ba5-a487-a08997b19367 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:33.536996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a7d2-cac1-4ba5-a487-a08997b19367 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:33.537276 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a7d2-cac1-4ba5-a487-a08997b19367 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:33.538592 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e5bed-8345-4000-8519-685620f98347 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:33.540037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7b51a7d2-cac1-4ba5-a487-a08997b19367 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:33.561442 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 835/1693] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:33.563351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b4576-f37d-45db-9ffd-6c130b90585d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:33.563351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b4576-f37d-45db-9ffd-6c130b90585d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:33.566028 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b4576-f37d-45db-9ffd-6c130b90585d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:33.608966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-296b4576-f37d-45db-9ffd-6c130b90585d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:33.612693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-296b4576-f37d-45db-9ffd-6c130b90585d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:33.639513 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 836/1694] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:33.640453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b4576-f37d-45db-9ffd-6c130b90585d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:33.658737 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b4576-f37d-45db-9ffd-6c130b90585d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:33.687071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456733b-6693-4705-8dc2-37702060af0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:33.688110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456733b-6693-4705-8dc2-37702060af0f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:33.688390 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456733b-6693-4705-8dc2-37702060af0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:33.688671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456733b-6693-4705-8dc2-37702060af0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:33.689426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456733b-6693-4705-8dc2-37702060af0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:33.689740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456733b-6693-4705-8dc2-37702060af0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:33.691477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456733b-6693-4705-8dc2-37702060af0f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:33.722633 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 837/1695] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:33.723630 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456733b-6693-4705-8dc2-37702060af0f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:33.743827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456733b-6693-4705-8dc2-37702060af0f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:33.769498 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51a7d2-cac1-4ba5-a487-a08997b19367 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:33.771338 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51a7d2-cac1-4ba5-a487-a08997b19367 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:33.776354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe54ce-9b3c-41d5-9b9c-70dc300c886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:33.777242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe54ce-9b3c-41d5-9b9c-70dc300c886f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:33.777696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe54ce-9b3c-41d5-9b9c-70dc300c886f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:33.778143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe54ce-9b3c-41d5-9b9c-70dc300c886f admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:33.779101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe54ce-9b3c-41d5-9b9c-70dc300c886f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:33.781263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe54ce-9b3c-41d5-9b9c-70dc300c886f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:33.835437 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 859/1696] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:33.840964 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 838/1697] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:33 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:33.845824 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a7d2-cac1-4ba5-a487-a08997b19367 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:33.902817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a7d2-cac1-4ba5-a487-a08997b19367 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:34.133423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac335e8d-aebb-4244-89eb-731f60cc7f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:34.134159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac335e8d-aebb-4244-89eb-731f60cc7f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:34.135882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac335e8d-aebb-4244-89eb-731f60cc7f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:34.135882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac335e8d-aebb-4244-89eb-731f60cc7f3f admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:34.136464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe54ce-9b3c-41d5-9b9c-70dc300c886f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:34.138405 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1dfe54ce-9b3c-41d5-9b9c-70dc300c886f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:34.138528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac335e8d-aebb-4244-89eb-731f60cc7f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:34.142624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac335e8d-aebb-4244-89eb-731f60cc7f3f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:34.157697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:34.161873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe54ce-9b3c-41d5-9b9c-70dc300c886f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:34.178348 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f9517-cfd8-46e9-bb5c-026bc3a26282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:34.179430 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f9517-cfd8-46e9-bb5c-026bc3a26282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:34.179815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f9517-cfd8-46e9-bb5c-026bc3a26282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:34.180193 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f9517-cfd8-46e9-bb5c-026bc3a26282 admin admin] PATH_INFO: `/v3/users/b63fb78fbcfa449db8c0034eb9e12449` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:34.181532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f9517-cfd8-46e9-bb5c-026bc3a26282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63fb78fbcfa449db8c0034eb9e12449)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:34.183598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f9517-cfd8-46e9-bb5c-026bc3a26282 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:34.186878 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 860/1698] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:33 2022] POST /identity/v3/users => generated 284 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:34.191673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac335e8d-aebb-4244-89eb-731f60cc7f3f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:34.212108 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac335e8d-aebb-4244-89eb-731f60cc7f3f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:34.218997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce157fce-e0a5-4df2-8e39-c2c6a41645a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:34.220412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce157fce-e0a5-4df2-8e39-c2c6a41645a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:34.220710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce157fce-e0a5-4df2-8e39-c2c6a41645a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:34.221061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce157fce-e0a5-4df2-8e39-c2c6a41645a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:34.222585 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce157fce-e0a5-4df2-8e39-c2c6a41645a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:34.225110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce157fce-e0a5-4df2-8e39-c2c6a41645a9 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:34.241371 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 861/1699] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:34.254873 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce157fce-e0a5-4df2-8e39-c2c6a41645a9 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:34.273046 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce157fce-e0a5-4df2-8e39-c2c6a41645a9 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:34.287513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab7a6-a757-4f6e-8551-0f4539473251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:34.292507 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-060f9517-cfd8-46e9-bb5c-026bc3a26282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63fb78fbcfa449db8c0034eb9e12449'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:34.295860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-060f9517-cfd8-46e9-bb5c-026bc3a26282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63fb78fbcfa449db8c0034eb9e12449'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:34.298253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-060f9517-cfd8-46e9-bb5c-026bc3a26282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63fb78fbcfa449db8c0034eb9e12449'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:34.303551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab7a6-a757-4f6e-8551-0f4539473251 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:34.304118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab7a6-a757-4f6e-8551-0f4539473251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:34.305102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab7a6-a757-4f6e-8551-0f4539473251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:34.306062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ab7a6-a757-4f6e-8551-0f4539473251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:34.308545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ab7a6-a757-4f6e-8551-0f4539473251 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:34.327078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-060f9517-cfd8-46e9-bb5c-026bc3a26282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63fb78fbcfa449db8c0034eb9e12449'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:34.328120 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 862/1700] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:34.331941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab7a6-a757-4f6e-8551-0f4539473251 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:34.334626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ff5ab7a6-a757-4f6e-8551-0f4539473251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:34.370480 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-060f9517-cfd8-46e9-bb5c-026bc3a26282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63fb78fbcfa449db8c0034eb9e12449'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:34.381975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab7a6-a757-4f6e-8551-0f4539473251 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:34.389685 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19184e6-c68e-4231-ab3e-23404e8fb7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:34.391771 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19184e6-c68e-4231-ab3e-23404e8fb7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:34.392401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19184e6-c68e-4231-ab3e-23404e8fb7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:34.392877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19184e6-c68e-4231-ab3e-23404e8fb7ab admin admin] PATH_INFO: `/v3/projects/aa91abd5cadb4bd9b7ace631450a8d1b/users/3bb7ea35e1964544b538036ba4ed3457/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:34.397217 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:34.423859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19184e6-c68e-4231-ab3e-23404e8fb7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa91abd5cadb4bd9b7ace631450a8d1b, user_id=3bb7ea35e1964544b538036ba4ed3457, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:34.428267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19184e6-c68e-4231-ab3e-23404e8fb7ab admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:34.439971 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 839/1701] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:34 2022] DELETE /identity/v3/users/b63fb78fbcfa449db8c0034eb9e12449 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:34.444844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e66023-87b7-4321-a47e-bdda37c04623 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:34.445159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e66023-87b7-4321-a47e-bdda37c04623 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:34.445429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e66023-87b7-4321-a47e-bdda37c04623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:34.458833 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 863/1702] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:34 2022] PUT /identity/v3/projects/aa91abd5cadb4bd9b7ace631450a8d1b/users/3bb7ea35e1964544b538036ba4ed3457/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:12:34.473715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ab389-a313-4dcd-a864-523b04423600 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:34.474356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ab389-a313-4dcd-a864-523b04423600 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:34.474887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ab389-a313-4dcd-a864-523b04423600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:34.508432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-78e66023-87b7-4321-a47e-bdda37c04623 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:34.509755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-78e66023-87b7-4321-a47e-bdda37c04623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:34.579434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-af7ab389-a313-4dcd-a864-523b04423600 None None] MFA Rules not processed for user `3bb7ea35e1964544b538036ba4ed3457`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:34.582011 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-af7ab389-a313-4dcd-a864-523b04423600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:34.681480 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 840/1703] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:34.698414 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e66023-87b7-4321-a47e-bdda37c04623 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:34.726760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-78e66023-87b7-4321-a47e-bdda37c04623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:34.783784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e66023-87b7-4321-a47e-bdda37c04623 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:34.805101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:34.808915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:34.816410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:34.816410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] PATH_INFO: `/v3/projects/e545a9dca63a4f0aa719494f273262af` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:34.816614 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 864/1704] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:34.819919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ab389-a313-4dcd-a864-523b04423600 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:34.820043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e545a9dca63a4f0aa719494f273262af)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:34.826154 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:34.892148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ab389-a313-4dcd-a864-523b04423600 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:34.908749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a0a89-2fa7-4f77-b8a5-197542d1f62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:34.909442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a0a89-2fa7-4f77-b8a5-197542d1f62b admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:34.909728 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a0a89-2fa7-4f77-b8a5-197542d1f62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:34.910004 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a0a89-2fa7-4f77-b8a5-197542d1f62b admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:34.910744 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a0a89-2fa7-4f77-b8a5-197542d1f62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:34.912857 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a0a89-2fa7-4f77-b8a5-197542d1f62b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:34.973998 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 865/1705] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:34 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:34.975024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a0a89-2fa7-4f77-b8a5-197542d1f62b admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:34.983498 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e28a0a89-2fa7-4f77-b8a5-197542d1f62b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:34.985693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e545a9dca63a4f0aa719494f273262af', 'request_id': 'req-c0875bb6-2725-48eb-83e6-89d082018817'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:34.999292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e545a9dca63a4f0aa719494f273262af', 'request_id': 'req-c0875bb6-2725-48eb-83e6-89d082018817'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:35.029066 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e545a9dca63a4f0aa719494f273262af', 'request_id': 'req-c0875bb6-2725-48eb-83e6-89d082018817'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:35.061371 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:35.093895 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] The token cache is being invalidate because project e545a9dca63a4f0aa719494f273262af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:35.095455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:35.097650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:35.099992 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 841/1706] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:34 2022] DELETE /identity/v3/projects/e545a9dca63a4f0aa719494f273262af => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:35.111549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:35.113404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:35.117431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a0a89-2fa7-4f77-b8a5-197542d1f62b admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:35.130391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c470-5fcd-4810-b62d-119730e90c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:35.131349 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c470-5fcd-4810-b62d-119730e90c13 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:35.131733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c470-5fcd-4810-b62d-119730e90c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:35.132109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c470-5fcd-4810-b62d-119730e90c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:35.137349 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ce44c470-5fcd-4810-b62d-119730e90c13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:35.332947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0875bb6-2725-48eb-83e6-89d082018817 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:35.337790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44c470-5fcd-4810-b62d-119730e90c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:35.346419 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44c470-5fcd-4810-b62d-119730e90c13 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:35.353088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:35.398770 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 866/1707] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:35.403174 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c470-5fcd-4810-b62d-119730e90c13 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:35.407716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ce44c470-5fcd-4810-b62d-119730e90c13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:35.454926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c470-5fcd-4810-b62d-119730e90c13 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:35.483514 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a9f24-e140-43df-8d5d-d708165e16d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:35.487410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a9f24-e140-43df-8d5d-d708165e16d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:35.487816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a9f24-e140-43df-8d5d-d708165e16d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:35.489174 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a9f24-e140-43df-8d5d-d708165e16d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:35.491171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a9f24-e140-43df-8d5d-d708165e16d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:35.495314 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a9f24-e140-43df-8d5d-d708165e16d9 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:35.565866 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 867/1708] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:35 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:35.580611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45ae3f-60e0-4007-99fc-07c3409fd509 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:35.584822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45ae3f-60e0-4007-99fc-07c3409fd509 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:35.587142 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45ae3f-60e0-4007-99fc-07c3409fd509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:35.607523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40f1a6-e91e-4d42-9ce5-29fe5d789567 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:35.608323 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f1a6-e91e-4d42-9ce5-29fe5d789567 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:35.614565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f1a6-e91e-4d42-9ce5-29fe5d789567 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:35.616560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f1a6-e91e-4d42-9ce5-29fe5d789567 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:35.641761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9a40f1a6-e91e-4d42-9ce5-29fe5d789567 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:35.669628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-fd45ae3f-60e0-4007-99fc-07c3409fd509 None None] MFA Rules not processed for user `3c6e5faddabd4208b6e476be4b59e834`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:35.676418 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-fd45ae3f-60e0-4007-99fc-07c3409fd509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:35.699270 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40f1a6-e91e-4d42-9ce5-29fe5d789567 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:35.705155 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40f1a6-e91e-4d42-9ce5-29fe5d789567 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:35.747019 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 842/1709] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:35.749238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfae30-a0e8-47aa-b33d-de1883a4823f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:35.758215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfae30-a0e8-47aa-b33d-de1883a4823f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:35.759247 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfae30-a0e8-47aa-b33d-de1883a4823f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:35.820478 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-41cfae30-a0e8-47aa-b33d-de1883a4823f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:35.822598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-41cfae30-a0e8-47aa-b33d-de1883a4823f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:35.854730 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 843/1710] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:35.854730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfae30-a0e8-47aa-b33d-de1883a4823f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:35.879398 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 868/1711] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:35.880344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfae30-a0e8-47aa-b33d-de1883a4823f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:35.892688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746194f3-0bf8-422d-9bf9-8beb7fe3c033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:35.892688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746194f3-0bf8-422d-9bf9-8beb7fe3c033 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:35.892688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746194f3-0bf8-422d-9bf9-8beb7fe3c033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:35.897283 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746194f3-0bf8-422d-9bf9-8beb7fe3c033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:35.898306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746194f3-0bf8-422d-9bf9-8beb7fe3c033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:35.899994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746194f3-0bf8-422d-9bf9-8beb7fe3c033 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:35.900938 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c9139-f901-43ca-a77f-0ab5ea2ef298 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:35.901336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c9139-f901-43ca-a77f-0ab5ea2ef298 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:35.901733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c9139-f901-43ca-a77f-0ab5ea2ef298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:35.929864 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 844/1712] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:35.930907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746194f3-0bf8-422d-9bf9-8beb7fe3c033 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:35.966004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746194f3-0bf8-422d-9bf9-8beb7fe3c033 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:35.978469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-2e0c9139-f901-43ca-a77f-0ab5ea2ef298 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:35.979936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2e0c9139-f901-43ca-a77f-0ab5ea2ef298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:35.986958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b13b2e-cbf1-4020-b47a-fd0de3751846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:35.990207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13b2e-cbf1-4020-b47a-fd0de3751846 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:35.993034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13b2e-cbf1-4020-b47a-fd0de3751846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:35.993884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13b2e-cbf1-4020-b47a-fd0de3751846 admin admin] PATH_INFO: `/v3/projects/751e004e76d74415a25f2c5cb0969516/users/cc43aa71d0294f9aa31daace8053a93e/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:36.003396 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 869/1713] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:36.004532 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c9139-f901-43ca-a77f-0ab5ea2ef298 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:36.032868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2e0c9139-f901-43ca-a77f-0ab5ea2ef298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:36.045455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b13b2e-cbf1-4020-b47a-fd0de3751846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=751e004e76d74415a25f2c5cb0969516, user_id=cc43aa71d0294f9aa31daace8053a93e, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:36.057278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b13b2e-cbf1-4020-b47a-fd0de3751846 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:36.103667 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 845/1714] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:35 2022] PUT /identity/v3/projects/751e004e76d74415a25f2c5cb0969516/users/cc43aa71d0294f9aa31daace8053a93e/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:36.103667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b13b2e-cbf1-4020-b47a-fd0de3751846 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:36.145467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c9139-f901-43ca-a77f-0ab5ea2ef298 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:36.217894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b5267-d0d2-44a3-8d75-d1927579c647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:36.219173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b5267-d0d2-44a3-8d75-d1927579c647 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:36.224029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b5267-d0d2-44a3-8d75-d1927579c647 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:36.224636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b5267-d0d2-44a3-8d75-d1927579c647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:36.227105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3f1b5267-d0d2-44a3-8d75-d1927579c647 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:36.274436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:36.298835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b13b2e-cbf1-4020-b47a-fd0de3751846 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:36.315964 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035aaa4-8e18-4f0b-815f-60b95e06afc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:36.316699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035aaa4-8e18-4f0b-815f-60b95e06afc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:36.316987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035aaa4-8e18-4f0b-815f-60b95e06afc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:36.317271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035aaa4-8e18-4f0b-815f-60b95e06afc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:36.318037 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035aaa4-8e18-4f0b-815f-60b95e06afc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:36.318382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035aaa4-8e18-4f0b-815f-60b95e06afc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:36.320705 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035aaa4-8e18-4f0b-815f-60b95e06afc7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:36.351227 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 846/1715] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:36.352960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035aaa4-8e18-4f0b-815f-60b95e06afc7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:36.369570 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035aaa4-8e18-4f0b-815f-60b95e06afc7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:36.376311 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:36.380345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b471b3d-fbb3-4088-a653-22c187a98071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:36.381256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b471b3d-fbb3-4088-a653-22c187a98071 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:36.382134 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b471b3d-fbb3-4088-a653-22c187a98071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:36.382586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b471b3d-fbb3-4088-a653-22c187a98071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:36.384250 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b471b3d-fbb3-4088-a653-22c187a98071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:36.386648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b471b3d-fbb3-4088-a653-22c187a98071 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:36.413469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b5267-d0d2-44a3-8d75-d1927579c647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:36.421138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b5267-d0d2-44a3-8d75-d1927579c647 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:36.434432 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 847/1716] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:36.436884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd29f9-9ae5-401c-8007-54aa77dff19d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:36.438545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd29f9-9ae5-401c-8007-54aa77dff19d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:36.438966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd29f9-9ae5-401c-8007-54aa77dff19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:36.480825 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 870/1717] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:36.483183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b5267-d0d2-44a3-8d75-d1927579c647 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:36.529924 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b5267-d0d2-44a3-8d75-d1927579c647 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:36.547269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f8544-44c5-4ec0-a4e5-51186dba65fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:36.547983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f8544-44c5-4ec0-a4e5-51186dba65fc admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:36.548270 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f8544-44c5-4ec0-a4e5-51186dba65fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:36.548544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f8544-44c5-4ec0-a4e5-51186dba65fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:36.558157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f8544-44c5-4ec0-a4e5-51186dba65fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:36.558157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f8544-44c5-4ec0-a4e5-51186dba65fc admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:36.573131 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6cfd29f9-9ae5-401c-8007-54aa77dff19d None None] MFA Rules not processed for user `7d4bb910a5bc4c7ea2667dc65066b2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:36.575612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6cfd29f9-9ae5-401c-8007-54aa77dff19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:36.628444 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 871/1718] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:36 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:36.629806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f8544-44c5-4ec0-a4e5-51186dba65fc admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:36.787753 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f8544-44c5-4ec0-a4e5-51186dba65fc admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:36.803855 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f471b7-e1c8-4323-bc73-5d6ea045e2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:36.805231 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f471b7-e1c8-4323-bc73-5d6ea045e2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:36.806521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f471b7-e1c8-4323-bc73-5d6ea045e2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:36.806818 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f471b7-e1c8-4323-bc73-5d6ea045e2e0 admin admin] PATH_INFO: `/v3/projects/751e004e76d74415a25f2c5cb0969516/users/cc43aa71d0294f9aa31daace8053a93e/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:36.813868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f471b7-e1c8-4323-bc73-5d6ea045e2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=751e004e76d74415a25f2c5cb0969516, user_id=cc43aa71d0294f9aa31daace8053a93e, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:36.824121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f471b7-e1c8-4323-bc73-5d6ea045e2e0 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:36.852850 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 872/1720] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:36 2022] PUT /identity/v3/projects/751e004e76d74415a25f2c5cb0969516/users/cc43aa71d0294f9aa31daace8053a93e/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:36.853216 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 848/1720] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:36.855897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f471b7-e1c8-4323-bc73-5d6ea045e2e0 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:36.864032 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caafd4-e720-41d8-9ea8-dffe17bb716f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:36.864615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caafd4-e720-41d8-9ea8-dffe17bb716f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:36.865088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caafd4-e720-41d8-9ea8-dffe17bb716f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:36.933087 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f3caafd4-e720-41d8-9ea8-dffe17bb716f None None] MFA Rules not processed for user `7d4bb910a5bc4c7ea2667dc65066b2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:36.936359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f3caafd4-e720-41d8-9ea8-dffe17bb716f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:37.077217 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f471b7-e1c8-4323-bc73-5d6ea045e2e0 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:37.085357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff934fa2-8042-450d-acb3-d50952698504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:37.086945 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff934fa2-8042-450d-acb3-d50952698504 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:37.087509 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff934fa2-8042-450d-acb3-d50952698504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:37.087921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff934fa2-8042-450d-acb3-d50952698504 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:37.088825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff934fa2-8042-450d-acb3-d50952698504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:37.093451 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff934fa2-8042-450d-acb3-d50952698504 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:37.126820 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 849/1721] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:37.126820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593df1a-f1c3-4566-8181-b4e7db8b0b00 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:37.126820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593df1a-f1c3-4566-8181-b4e7db8b0b00 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:37.126820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593df1a-f1c3-4566-8181-b4e7db8b0b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:37.173148 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 873/1722] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:36 2022] POST /identity/v3/users => generated 290 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:37.174241 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff934fa2-8042-450d-acb3-d50952698504 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:37.179225 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ff934fa2-8042-450d-acb3-d50952698504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:37.214200 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8593df1a-f1c3-4566-8181-b4e7db8b0b00 None None] MFA Rules not processed for user `cc43aa71d0294f9aa31daace8053a93e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:37.217515 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff934fa2-8042-450d-acb3-d50952698504 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:37.220004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8593df1a-f1c3-4566-8181-b4e7db8b0b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:37.233978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:37.235337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:37.235741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:37.236191 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] PATH_INFO: `/v3/projects/227f2bd5d6d44255ad34e052b505d42c` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:37.244876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] RBAC: Authorizing `identity:delete_project(project_id=227f2bd5d6d44255ad34e052b505d42c)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:37.252206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:37.364734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:37.387720 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:37.388345 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 850/1723] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:37.389917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593df1a-f1c3-4566-8181-b4e7db8b0b00 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:37.392215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8593df1a-f1c3-4566-8181-b4e7db8b0b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:37.407283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '227f2bd5d6d44255ad34e052b505d42c', 'request_id': 'req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:37.422914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '227f2bd5d6d44255ad34e052b505d42c', 'request_id': 'req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:37.439901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '227f2bd5d6d44255ad34e052b505d42c', 'request_id': 'req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:37.472336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] The token cache is being invalidate because project 227f2bd5d6d44255ad34e052b505d42c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:37.473160 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:37.474404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:37.492563 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 874/1724] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:37 2022] DELETE /identity/v3/projects/227f2bd5d6d44255ad34e052b505d42c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:37.494027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:37.495796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:37.543017 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593df1a-f1c3-4566-8181-b4e7db8b0b00 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:37.617457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797a9fb-7f5f-4a2d-98bc-38f5f7689152 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:37.623425 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2780be-0701-4d45-abad-3d963e0c4600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:37.624239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2780be-0701-4d45-abad-3d963e0c4600 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:37.624553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2780be-0701-4d45-abad-3d963e0c4600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:37.624841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2780be-0701-4d45-abad-3d963e0c4600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:37.627634 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2780be-0701-4d45-abad-3d963e0c4600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:37.629596 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2780be-0701-4d45-abad-3d963e0c4600 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:37.643107 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 875/1725] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:37.643975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2780be-0701-4d45-abad-3d963e0c4600 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:37.646006 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-df2780be-0701-4d45-abad-3d963e0c4600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:37.698753 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c121d-913e-4f53-9b44-0bb8168d2576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:37.699446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c121d-913e-4f53-9b44-0bb8168d2576 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:37.699729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c121d-913e-4f53-9b44-0bb8168d2576 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:37.700254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c121d-913e-4f53-9b44-0bb8168d2576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:37.709161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ac1c121d-913e-4f53-9b44-0bb8168d2576 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:37.728190 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2780be-0701-4d45-abad-3d963e0c4600 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:37.737291 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976793d2-b3bb-4055-8e66-f5a7c39f3e45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:37.739425 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976793d2-b3bb-4055-8e66-f5a7c39f3e45 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:37.739858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976793d2-b3bb-4055-8e66-f5a7c39f3e45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:37.740141 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976793d2-b3bb-4055-8e66-f5a7c39f3e45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:37.744384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-976793d2-b3bb-4055-8e66-f5a7c39f3e45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:37.887154 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c121d-913e-4f53-9b44-0bb8168d2576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:37.888825 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c121d-913e-4f53-9b44-0bb8168d2576 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:37.903434 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 851/1726] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:37.904786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49493ced-7783-44d6-bc82-a89c9605aba0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:37.905077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49493ced-7783-44d6-bc82-a89c9605aba0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:37.905354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49493ced-7783-44d6-bc82-a89c9605aba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:37.941966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976793d2-b3bb-4055-8e66-f5a7c39f3e45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:37.943716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976793d2-b3bb-4055-8e66-f5a7c39f3e45 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:37.963322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-49493ced-7783-44d6-bc82-a89c9605aba0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:37.964594 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-49493ced-7783-44d6-bc82-a89c9605aba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:37.973121 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 876/1727] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:37.974852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976793d2-b3bb-4055-8e66-f5a7c39f3e45 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:37.999793 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 852/1728] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:38.001836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ede7bd-d62f-4970-bca1-419641226047 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:38.001836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ede7bd-d62f-4970-bca1-419641226047 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:38.002054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ede7bd-d62f-4970-bca1-419641226047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:38.004960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976793d2-b3bb-4055-8e66-f5a7c39f3e45 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:38.018813 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e02576-5983-40a7-9e05-340cb82369f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:38.019900 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e02576-5983-40a7-9e05-340cb82369f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:38.020099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e02576-5983-40a7-9e05-340cb82369f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:38.020696 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e02576-5983-40a7-9e05-340cb82369f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:38.021233 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e02576-5983-40a7-9e05-340cb82369f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:38.025854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e02576-5983-40a7-9e05-340cb82369f4 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:38.060703 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 877/1729] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:38.065963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e25880-89da-4071-83a6-1afdfba7333d None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:38.066238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b4ede7bd-d62f-4970-bca1-419641226047 None None] MFA Rules not processed for user `51e20b9a489342d2b8f0f85542ab370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:38.068395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e25880-89da-4071-83a6-1afdfba7333d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:38.068697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e25880-89da-4071-83a6-1afdfba7333d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:38.069879 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b4ede7bd-d62f-4970-bca1-419641226047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:38.102461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:38.103589 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:38.127660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-75e25880-89da-4071-83a6-1afdfba7333d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:38.128901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-75e25880-89da-4071-83a6-1afdfba7333d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:38.187827 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 878/1730] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:38.192424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e25880-89da-4071-83a6-1afdfba7333d None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:38.235337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e25880-89da-4071-83a6-1afdfba7333d None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:38.252966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe350e6-eb44-4e91-a3e9-b369b3aad547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:38.253917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe350e6-eb44-4e91-a3e9-b369b3aad547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:38.254196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe350e6-eb44-4e91-a3e9-b369b3aad547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:38.254505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe350e6-eb44-4e91-a3e9-b369b3aad547 admin admin] PATH_INFO: `/v3/projects/0b91f9ac0a1443e2a7fc986e9464caa4/users/47f0474641244fbfbac0fed8c83b8e1b/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:38.306553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe350e6-eb44-4e91-a3e9-b369b3aad547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b91f9ac0a1443e2a7fc986e9464caa4, user_id=47f0474641244fbfbac0fed8c83b8e1b, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:38.309316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe350e6-eb44-4e91-a3e9-b369b3aad547 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:38.313713 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 853/1731] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:38.319837 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ede7bd-d62f-4970-bca1-419641226047 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:38.332812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ede7bd-d62f-4970-bca1-419641226047 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:38.338779 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 879/1732] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:38 2022] PUT /identity/v3/projects/0b91f9ac0a1443e2a7fc986e9464caa4/users/47f0474641244fbfbac0fed8c83b8e1b/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:38.339809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe350e6-eb44-4e91-a3e9-b369b3aad547 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:38.495021 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d74f1c-7af3-438f-b555-bc9741baa2b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:38.495712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d74f1c-7af3-438f-b555-bc9741baa2b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:38.495997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d74f1c-7af3-438f-b555-bc9741baa2b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:38.496275 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d74f1c-7af3-438f-b555-bc9741baa2b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:38.501241 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-00d74f1c-7af3-438f-b555-bc9741baa2b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:38.614561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe350e6-eb44-4e91-a3e9-b369b3aad547 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:38.640246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234e45d-fe37-478c-9be9-816b1edbfd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:38.640990 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234e45d-fe37-478c-9be9-816b1edbfd56 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:38.641277 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234e45d-fe37-478c-9be9-816b1edbfd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:38.641554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234e45d-fe37-478c-9be9-816b1edbfd56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:38.642346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234e45d-fe37-478c-9be9-816b1edbfd56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:38.648080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234e45d-fe37-478c-9be9-816b1edbfd56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:38.650197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234e45d-fe37-478c-9be9-816b1edbfd56 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:38.682936 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 880/1733] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:38.701590 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731592c-b36b-4da4-9eb0-87977ecea0ea None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:38.701958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731592c-b36b-4da4-9eb0-87977ecea0ea None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:38.702256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731592c-b36b-4da4-9eb0-87977ecea0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:38.751607 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d74f1c-7af3-438f-b555-bc9741baa2b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:38.753067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d74f1c-7af3-438f-b555-bc9741baa2b2 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:38.809468 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 854/1734] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:38.810746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d74f1c-7af3-438f-b555-bc9741baa2b2 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:38.813271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-00d74f1c-7af3-438f-b555-bc9741baa2b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:38.817651 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-5731592c-b36b-4da4-9eb0-87977ecea0ea None None] MFA Rules not processed for user `47f0474641244fbfbac0fed8c83b8e1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:38.829436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5731592c-b36b-4da4-9eb0-87977ecea0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:39.112005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:39.132787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d74f1c-7af3-438f-b555-bc9741baa2b2 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:39.148488 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 881/1735] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:39.149864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731592c-b36b-4da4-9eb0-87977ecea0ea None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:39.161425 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678f004-6050-45bd-94aa-4c24f8c10ee8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:39.165074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678f004-6050-45bd-94aa-4c24f8c10ee8 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:39.165578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678f004-6050-45bd-94aa-4c24f8c10ee8 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:39.167369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678f004-6050-45bd-94aa-4c24f8c10ee8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:39.195496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731592c-b36b-4da4-9eb0-87977ecea0ea None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:39.220554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4f9fd-4244-40ba-86b8-4c313d859277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:39.221779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4f9fd-4244-40ba-86b8-4c313d859277 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:39.222292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4f9fd-4244-40ba-86b8-4c313d859277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:39.222767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4f9fd-4244-40ba-86b8-4c313d859277 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:39.223671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4f9fd-4244-40ba-86b8-4c313d859277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:39.225841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4f9fd-4244-40ba-86b8-4c313d859277 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:39.335057 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 882/1736] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:39 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:39.337588 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0807f-caa0-4100-832c-6f85572cdce1 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:39.340783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0807f-caa0-4100-832c-6f85572cdce1 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:39.341484 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0807f-caa0-4100-832c-6f85572cdce1 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:39.352677 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 883/1737] 10.222.0.16 () {54 vars in 912 bytes} [Tue Jul 5 10:12:39 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:12:39.353956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0807f-caa0-4100-832c-6f85572cdce1 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:39.438581 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678f004-6050-45bd-94aa-4c24f8c10ee8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:39.441441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678f004-6050-45bd-94aa-4c24f8c10ee8 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:39.486491 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 855/1738] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:39.487178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678f004-6050-45bd-94aa-4c24f8c10ee8 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:39.516057 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:39.538153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678f004-6050-45bd-94aa-4c24f8c10ee8 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:39.608206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0807f-caa0-4100-832c-6f85572cdce1 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:39.620695 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde2b33-9460-4e0b-bf87-b702b8333968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:39.621557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde2b33-9460-4e0b-bf87-b702b8333968 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:39.621838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde2b33-9460-4e0b-bf87-b702b8333968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:39.622106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde2b33-9460-4e0b-bf87-b702b8333968 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:39.622982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde2b33-9460-4e0b-bf87-b702b8333968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:39.624927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde2b33-9460-4e0b-bf87-b702b8333968 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:39.799330 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 884/1739] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:39 2022] POST /identity/v3/projects => generated 377 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:39.810371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207e1e8-7c73-4292-a10f-831faaba049c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:39.813546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207e1e8-7c73-4292-a10f-831faaba049c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:39.813842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207e1e8-7c73-4292-a10f-831faaba049c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:39.848664 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a1ea8-e08c-441f-bfcd-25ba66865c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:39.849494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a1ea8-e08c-441f-bfcd-25ba66865c37 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:39.849777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a1ea8-e08c-441f-bfcd-25ba66865c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:39.860655 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a1ea8-e08c-441f-bfcd-25ba66865c37 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:39.861484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a1ea8-e08c-441f-bfcd-25ba66865c37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:39.866926 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a1ea8-e08c-441f-bfcd-25ba66865c37 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:39.918336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-1207e1e8-7c73-4292-a10f-831faaba049c None None] MFA Rules not processed for user `e0865f21da0a4c95a84c0099a1b9a78b`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:39.919902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1207e1e8-7c73-4292-a10f-831faaba049c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:39.972653 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 856/1740] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:39 2022] POST /identity/v3/users => generated 283 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:39.983076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29e2f-b255-4972-b9e9-db4f5156da82 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:39.988148 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29e2f-b255-4972-b9e9-db4f5156da82 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:39.990586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29e2f-b255-4972-b9e9-db4f5156da82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:40.089258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-23c29e2f-b255-4972-b9e9-db4f5156da82 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:40.089258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-23c29e2f-b255-4972-b9e9-db4f5156da82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:40.122291 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 885/1741] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:12:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:40.123092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207e1e8-7c73-4292-a10f-831faaba049c None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:40.387486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207e1e8-7c73-4292-a10f-831faaba049c None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:40.390194 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 857/1742] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:12:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:40.391774 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c29e2f-b255-4972-b9e9-db4f5156da82 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:40.412412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6108cb9-36a4-4484-87b8-f2ff91e4fe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:40.416298 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6108cb9-36a4-4484-87b8-f2ff91e4fe2e admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:40.417792 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6108cb9-36a4-4484-87b8-f2ff91e4fe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:40.422600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6108cb9-36a4-4484-87b8-f2ff91e4fe2e admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:40.423583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6108cb9-36a4-4484-87b8-f2ff91e4fe2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:40.435081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6108cb9-36a4-4484-87b8-f2ff91e4fe2e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:40.451703 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c29e2f-b255-4972-b9e9-db4f5156da82 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:40.481594 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e79a02-34ba-46ec-b649-db99d2e0bf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:40.486991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e79a02-34ba-46ec-b649-db99d2e0bf21 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:40.490365 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e79a02-34ba-46ec-b649-db99d2e0bf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:40.491892 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e79a02-34ba-46ec-b649-db99d2e0bf21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:40.501959 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e79a02-34ba-46ec-b649-db99d2e0bf21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:40.509336 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 886/1743] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:40 2022] POST /identity/v3/users => generated 289 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:40.510468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6108cb9-36a4-4484-87b8-f2ff91e4fe2e admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:40.510582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e79a02-34ba-46ec-b649-db99d2e0bf21 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:40.517681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f6108cb9-36a4-4484-87b8-f2ff91e4fe2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:40.545317 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6108cb9-36a4-4484-87b8-f2ff91e4fe2e admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:40.558696 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8f7e3-d8cf-4878-90f1-177c3ab93774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:40.560342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8f7e3-d8cf-4878-90f1-177c3ab93774 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:40.560864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8f7e3-d8cf-4878-90f1-177c3ab93774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:40.561503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8f7e3-d8cf-4878-90f1-177c3ab93774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:40.564955 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 858/1744] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:40.564955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e79a02-34ba-46ec-b649-db99d2e0bf21 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:40.582498 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8f7e3-d8cf-4878-90f1-177c3ab93774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:40.588506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8f7e3-d8cf-4878-90f1-177c3ab93774 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:40.610761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e79a02-34ba-46ec-b649-db99d2e0bf21 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:40.612627 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 887/1745] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:40.613779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8f7e3-d8cf-4878-90f1-177c3ab93774 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:40.625160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67473d35-8241-440b-9c8f-02c1547a58cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:40.626212 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473d35-8241-440b-9c8f-02c1547a58cd admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:40.626528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473d35-8241-440b-9c8f-02c1547a58cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:40.626857 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473d35-8241-440b-9c8f-02c1547a58cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:40.632233 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67473d35-8241-440b-9c8f-02c1547a58cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:40.633447 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67473d35-8241-440b-9c8f-02c1547a58cd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:40.644369 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8f7e3-d8cf-4878-90f1-177c3ab93774 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:40.668398 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 859/1746] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:40.670762 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67473d35-8241-440b-9c8f-02c1547a58cd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:40.684210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3b3d1-7a08-4b80-bad6-23041127d828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:40.685356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3b3d1-7a08-4b80-bad6-23041127d828 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:40.685752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3b3d1-7a08-4b80-bad6-23041127d828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:40.686140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3b3d1-7a08-4b80-bad6-23041127d828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:40.687136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3b3d1-7a08-4b80-bad6-23041127d828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:40.689295 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3b3d1-7a08-4b80-bad6-23041127d828 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:40.700014 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67473d35-8241-440b-9c8f-02c1547a58cd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:40.711579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63830a-cf8b-4965-a923-bdfe6460b88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:40.712528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63830a-cf8b-4965-a923-bdfe6460b88f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:40.712819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63830a-cf8b-4965-a923-bdfe6460b88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:40.713154 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63830a-cf8b-4965-a923-bdfe6460b88f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:40.714149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63830a-cf8b-4965-a923-bdfe6460b88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:40.715860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63830a-cf8b-4965-a923-bdfe6460b88f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:40.735617 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 888/1747] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:40.748976 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 860/1748] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:40.750211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63830a-cf8b-4965-a923-bdfe6460b88f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:40.752223 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3b3d1-7a08-4b80-bad6-23041127d828 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:40.765146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:40.765491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c1d3b3d1-7a08-4b80-bad6-23041127d828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:40.774387 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63830a-cf8b-4965-a923-bdfe6460b88f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:40.785961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8d98e-1662-45bd-8ce7-9a574b37f8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:40.787154 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8d98e-1662-45bd-8ce7-9a574b37f8d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:40.792881 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8d98e-1662-45bd-8ce7-9a574b37f8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:40.793178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8d98e-1662-45bd-8ce7-9a574b37f8d3 admin admin] PATH_INFO: `/v3/projects/d802936f8ae940ea86161b4b3bc2e195/users/017ad6c76196440fa95f2e38f65d0e26/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:40.849474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8d98e-1662-45bd-8ce7-9a574b37f8d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d802936f8ae940ea86161b4b3bc2e195, user_id=017ad6c76196440fa95f2e38f65d0e26, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:40.864336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8d98e-1662-45bd-8ce7-9a574b37f8d3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:40.914321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:40.943536 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 861/1749] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:40 2022] PUT /identity/v3/projects/d802936f8ae940ea86161b4b3bc2e195/users/017ad6c76196440fa95f2e38f65d0e26/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:40.944167 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8d98e-1662-45bd-8ce7-9a574b37f8d3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:41.008960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3b3d1-7a08-4b80-bad6-23041127d828 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:41.204554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8d98e-1662-45bd-8ce7-9a574b37f8d3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:41.222678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91450a74-c6ef-479f-af27-1855112018d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:41.223526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91450a74-c6ef-479f-af27-1855112018d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:41.223805 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91450a74-c6ef-479f-af27-1855112018d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:41.224114 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91450a74-c6ef-479f-af27-1855112018d8 admin admin] PATH_INFO: `/v3/projects/eaa23b69daa541caa8d65f4fa976d0ef/users/77954d74b22244a3b8120402187cd14d/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:41.239637 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e398b-81f4-430c-be8a-eb9feedb091a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:41.240377 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e398b-81f4-430c-be8a-eb9feedb091a service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:41.240695 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e398b-81f4-430c-be8a-eb9feedb091a service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:41.240976 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e398b-81f4-430c-be8a-eb9feedb091a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:41.294862 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91450a74-c6ef-479f-af27-1855112018d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa23b69daa541caa8d65f4fa976d0ef, user_id=77954d74b22244a3b8120402187cd14d, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:41.310617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91450a74-c6ef-479f-af27-1855112018d8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:41.358083 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 862/1750] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:40 2022] PUT /identity/v3/projects/eaa23b69daa541caa8d65f4fa976d0ef/users/77954d74b22244a3b8120402187cd14d/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:41.368110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9e440-160e-4c6b-836a-aa8a5108f243 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:41.368410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9e440-160e-4c6b-836a-aa8a5108f243 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:41.368692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9e440-160e-4c6b-836a-aa8a5108f243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:41.472432 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e398b-81f4-430c-be8a-eb9feedb091a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:41.477132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e398b-81f4-430c-be8a-eb9feedb091a service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:41.501161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-78a9e440-160e-4c6b-836a-aa8a5108f243 None None] MFA Rules not processed for user `017ad6c76196440fa95f2e38f65d0e26`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:41.514621 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 889/1751] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:12:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 769 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:12:41.520527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-78a9e440-160e-4c6b-836a-aa8a5108f243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:41.526877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a507ec-239b-4446-a52e-99c3c1243c75 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:41.527166 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a507ec-239b-4446-a52e-99c3c1243c75 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:41.527444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a507ec-239b-4446-a52e-99c3c1243c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:41.763439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-49a507ec-239b-4446-a52e-99c3c1243c75 None None] MFA Rules not processed for user `77954d74b22244a3b8120402187cd14d`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:41.771907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-49a507ec-239b-4446-a52e-99c3c1243c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:41.854294 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 863/1752] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:41.855350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9e440-160e-4c6b-836a-aa8a5108f243 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:41.863699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-78a9e440-160e-4c6b-836a-aa8a5108f243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:42.124251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:42.183640 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9e440-160e-4c6b-836a-aa8a5108f243 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:42.195454 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 890/1753] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 680 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:42.196439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a507ec-239b-4446-a52e-99c3c1243c75 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:42.206411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:42.213767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:42.214061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:42.214374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] PATH_INFO: `/v3/users/33370bcfbc8c4bf4ac33a62897d9f513` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:42.216701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33370bcfbc8c4bf4ac33a62897d9f513)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:42.218706 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:42.239555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a507ec-239b-4446-a52e-99c3c1243c75 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:42.256681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77010089-b0f6-4e2a-9ed3-552e2c66d5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:42.257671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010089-b0f6-4e2a-9ed3-552e2c66d5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:42.260775 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010089-b0f6-4e2a-9ed3-552e2c66d5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:42.263473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010089-b0f6-4e2a-9ed3-552e2c66d5d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:42.264468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77010089-b0f6-4e2a-9ed3-552e2c66d5d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:42.267719 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77010089-b0f6-4e2a-9ed3-552e2c66d5d6 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:42.306642 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:42.365744 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 891/1754] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:42 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:42.366814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77010089-b0f6-4e2a-9ed3-552e2c66d5d6 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:42.383440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33370bcfbc8c4bf4ac33a62897d9f513'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:42.386705 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33370bcfbc8c4bf4ac33a62897d9f513'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:42.388729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33370bcfbc8c4bf4ac33a62897d9f513'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:42.412062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33370bcfbc8c4bf4ac33a62897d9f513'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:42.471255 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33370bcfbc8c4bf4ac33a62897d9f513'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:42.524247 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 864/1755] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:41 2022] DELETE /identity/v3/users/33370bcfbc8c4bf4ac33a62897d9f513 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:42.525184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:42.528770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:42.550185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77010089-b0f6-4e2a-9ed3-552e2c66d5d6 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:42.755009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567fb51-24ee-43ae-b548-586a1539d6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:42.756075 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567fb51-24ee-43ae-b548-586a1539d6dd admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:42.756538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567fb51-24ee-43ae-b548-586a1539d6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:42.756989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567fb51-24ee-43ae-b548-586a1539d6dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:42.776569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567fb51-24ee-43ae-b548-586a1539d6dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:42.780815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567fb51-24ee-43ae-b548-586a1539d6dd admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:42.839817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca618856-a8e0-4d33-82e4-44826897a239 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:42.863836 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 892/1756] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:42 2022] POST /identity/v3/projects => generated 377 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:42.865369 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab17000-a90e-4787-bece-1244b06a455b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:42.865712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab17000-a90e-4787-bece-1244b06a455b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:42.868451 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab17000-a90e-4787-bece-1244b06a455b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:42.869334 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae3070-838c-4fed-a8ac-6b5f93428c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:42.872386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae3070-838c-4fed-a8ac-6b5f93428c68 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:42.872704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae3070-838c-4fed-a8ac-6b5f93428c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:42.872953 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae3070-838c-4fed-a8ac-6b5f93428c68 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:42.875085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae3070-838c-4fed-a8ac-6b5f93428c68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:42.900132 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae3070-838c-4fed-a8ac-6b5f93428c68 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:42.913441 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-eab17000-a90e-4787-bece-1244b06a455b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:42.914795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-eab17000-a90e-4787-bece-1244b06a455b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:42.997077 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 865/1757] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:42 2022] POST /identity/v3/users => generated 289 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:42.998022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae3070-838c-4fed-a8ac-6b5f93428c68 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:43.010673 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-35ae3070-838c-4fed-a8ac-6b5f93428c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:43.049566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae3070-838c-4fed-a8ac-6b5f93428c68 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:43.164320 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 893/1758] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:43.165504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab17000-a90e-4787-bece-1244b06a455b None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:43.221246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab17000-a90e-4787-bece-1244b06a455b None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:43.231628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44da9a-f041-4e55-a234-ba672cf2a069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:43.233267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44da9a-f041-4e55-a234-ba672cf2a069 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:43.233763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44da9a-f041-4e55-a234-ba672cf2a069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:43.234176 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44da9a-f041-4e55-a234-ba672cf2a069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:43.235345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44da9a-f041-4e55-a234-ba672cf2a069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:43.237234 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44da9a-f041-4e55-a234-ba672cf2a069 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:43.262778 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 894/1759] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:43.264404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44da9a-f041-4e55-a234-ba672cf2a069 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:43.267981 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3c44da9a-f041-4e55-a234-ba672cf2a069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:43.288344 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:43.350613 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ec1ba-cf59-49b3-9cae-bda26ce58eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:43.354030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ec1ba-cf59-49b3-9cae-bda26ce58eec admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:43.354030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ec1ba-cf59-49b3-9cae-bda26ce58eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:43.354289 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ec1ba-cf59-49b3-9cae-bda26ce58eec admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:43.362586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ec1ba-cf59-49b3-9cae-bda26ce58eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:43.373838 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ec1ba-cf59-49b3-9cae-bda26ce58eec admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:43.417090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:43.464785 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 866/1760] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:42 2022] POST /identity/v3/users => generated 295 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:43.472792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ec1ba-cf59-49b3-9cae-bda26ce58eec admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:43.531533 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ec1ba-cf59-49b3-9cae-bda26ce58eec admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:43.615419 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b1a8-646e-417c-b9d3-409c6fad9761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:43.619848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b1a8-646e-417c-b9d3-409c6fad9761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:43.623716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b1a8-646e-417c-b9d3-409c6fad9761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:43.623716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b1a8-646e-417c-b9d3-409c6fad9761 admin admin] PATH_INFO: `/v3/projects/66a5947cfaa447e99a2aa0187ea7672a/users/5bc768ad1b43466f966b033b779660fb/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:43.706170 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208b1a8-646e-417c-b9d3-409c6fad9761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a5947cfaa447e99a2aa0187ea7672a, user_id=5bc768ad1b43466f966b033b779660fb, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:43.713806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208b1a8-646e-417c-b9d3-409c6fad9761 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:43.716249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44da9a-f041-4e55-a234-ba672cf2a069 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:43.728089 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58407d7d-6128-4bfe-a9e4-8c4c957339b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:43.728777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407d7d-6128-4bfe-a9e4-8c4c957339b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:43.729060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407d7d-6128-4bfe-a9e4-8c4c957339b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:43.731005 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407d7d-6128-4bfe-a9e4-8c4c957339b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:43.744039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58407d7d-6128-4bfe-a9e4-8c4c957339b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:43.745544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58407d7d-6128-4bfe-a9e4-8c4c957339b2 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:43.753916 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 867/1761] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:43 2022] PUT /identity/v3/projects/66a5947cfaa447e99a2aa0187ea7672a/users/5bc768ad1b43466f966b033b779660fb/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:43.755210 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b1a8-646e-417c-b9d3-409c6fad9761 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:43.799888 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 895/1762] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:43.802214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58407d7d-6128-4bfe-a9e4-8c4c957339b2 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:43.833040 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58407d7d-6128-4bfe-a9e4-8c4c957339b2 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:43.980405 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b1a8-646e-417c-b9d3-409c6fad9761 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:44.026861 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa779ff9-a30e-4ccb-9b2e-d3c5ff86d9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:44.027974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa779ff9-a30e-4ccb-9b2e-d3c5ff86d9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:44.028510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa779ff9-a30e-4ccb-9b2e-d3c5ff86d9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:44.028933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa779ff9-a30e-4ccb-9b2e-d3c5ff86d9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:44.030164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa779ff9-a30e-4ccb-9b2e-d3c5ff86d9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:44.034678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa779ff9-a30e-4ccb-9b2e-d3c5ff86d9ef admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:44.050381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63882465-8b83-4e75-a615-5526477ec905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:44.051126 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63882465-8b83-4e75-a615-5526477ec905 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:44.051599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63882465-8b83-4e75-a615-5526477ec905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:44.051762 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 868/1763] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:44.056715 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa779ff9-a30e-4ccb-9b2e-d3c5ff86d9ef admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:44.058209 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63882465-8b83-4e75-a615-5526477ec905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:44.059167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63882465-8b83-4e75-a615-5526477ec905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:44.061096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63882465-8b83-4e75-a615-5526477ec905 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:44.088141 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa779ff9-a30e-4ccb-9b2e-d3c5ff86d9ef admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:44.110101 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 896/1764] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:44.111925 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63882465-8b83-4e75-a615-5526477ec905 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:44.125823 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe30be-2a67-42c6-ab9c-ed318d63c488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:44.127456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe30be-2a67-42c6-ab9c-ed318d63c488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:44.130474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe30be-2a67-42c6-ab9c-ed318d63c488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:44.130976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe30be-2a67-42c6-ab9c-ed318d63c488 admin admin] PATH_INFO: `/v3/projects/24e10bfffb814ed2ad0de965b84ea08d/users/0796890e391a4ffeabb664e0e64c8b95/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:44.146564 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63882465-8b83-4e75-a615-5526477ec905 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:44.171307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d5838-3e00-452f-bd57-2a655673f60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:44.172158 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d5838-3e00-452f-bd57-2a655673f60a admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:44.172436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d5838-3e00-452f-bd57-2a655673f60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:44.172712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d5838-3e00-452f-bd57-2a655673f60a admin admin] PATH_INFO: `/v3/projects/66a5947cfaa447e99a2aa0187ea7672a/users/5bc768ad1b43466f966b033b779660fb/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:44.175312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d5838-3e00-452f-bd57-2a655673f60a admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a5947cfaa447e99a2aa0187ea7672a, user_id=5bc768ad1b43466f966b033b779660fb, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:44.186809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d5838-3e00-452f-bd57-2a655673f60a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:44.240838 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 897/1765] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:44 2022] PUT /identity/v3/projects/66a5947cfaa447e99a2aa0187ea7672a/users/5bc768ad1b43466f966b033b779660fb/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:44.258748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca89bc0-5fbb-428c-aa97-8f7906b4672d None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:44.259312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca89bc0-5fbb-428c-aa97-8f7906b4672d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:44.259608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca89bc0-5fbb-428c-aa97-8f7906b4672d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:44.260989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe30be-2a67-42c6-ab9c-ed318d63c488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e10bfffb814ed2ad0de965b84ea08d, user_id=0796890e391a4ffeabb664e0e64c8b95, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:44.266008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe30be-2a67-42c6-ab9c-ed318d63c488 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:44.319697 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 869/1766] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:44 2022] PUT /identity/v3/projects/24e10bfffb814ed2ad0de965b84ea08d/users/0796890e391a4ffeabb664e0e64c8b95/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:44.321009 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe30be-2a67-42c6-ab9c-ed318d63c488 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:44.362520 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-6ca89bc0-5fbb-428c-aa97-8f7906b4672d None None] MFA Rules not processed for user `5bc768ad1b43466f966b033b779660fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:44.369250 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6ca89bc0-5fbb-428c-aa97-8f7906b4672d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:44.629423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe30be-2a67-42c6-ab9c-ed318d63c488 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:44.649617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435334de-95ff-4fc5-8a13-ab07c13529b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:44.650444 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435334de-95ff-4fc5-8a13-ab07c13529b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:44.650713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435334de-95ff-4fc5-8a13-ab07c13529b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:44.650993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435334de-95ff-4fc5-8a13-ab07c13529b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:44.651757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435334de-95ff-4fc5-8a13-ab07c13529b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:44.665035 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435334de-95ff-4fc5-8a13-ab07c13529b4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:44.673796 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 898/1767] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:44.682042 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca89bc0-5fbb-428c-aa97-8f7906b4672d None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:44.684419 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6ca89bc0-5fbb-428c-aa97-8f7906b4672d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:44.701298 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 870/1768] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:44.705937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435334de-95ff-4fc5-8a13-ab07c13529b4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:44.749846 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435334de-95ff-4fc5-8a13-ab07c13529b4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:44.804047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ddd1d-925e-42d4-b14e-f6e99488b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:44.810924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ddd1d-925e-42d4-b14e-f6e99488b871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:44.811212 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ddd1d-925e-42d4-b14e-f6e99488b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:44.812572 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ddd1d-925e-42d4-b14e-f6e99488b871 admin admin] PATH_INFO: `/v3/projects/24e10bfffb814ed2ad0de965b84ea08d/users/0796890e391a4ffeabb664e0e64c8b95/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:44.819358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ddd1d-925e-42d4-b14e-f6e99488b871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e10bfffb814ed2ad0de965b84ea08d, user_id=0796890e391a4ffeabb664e0e64c8b95, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:44.824141 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:44.839792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ddd1d-925e-42d4-b14e-f6e99488b871 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:44.878819 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 871/1769] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:44 2022] PUT /identity/v3/projects/24e10bfffb814ed2ad0de965b84ea08d/users/0796890e391a4ffeabb664e0e64c8b95/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:44.889350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fa4b5-cb93-4053-a2c6-e55616b95366 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:44.889667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fa4b5-cb93-4053-a2c6-e55616b95366 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:44.889885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fa4b5-cb93-4053-a2c6-e55616b95366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:44.912842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca89bc0-5fbb-428c-aa97-8f7906b4672d None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:44.975075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-576fa4b5-cb93-4053-a2c6-e55616b95366 None None] MFA Rules not processed for user `0796890e391a4ffeabb664e0e64c8b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:44.981829 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-576fa4b5-cb93-4053-a2c6-e55616b95366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:45.026346 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:45.126183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4357f-46c1-488c-b1ff-0daa60060e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:45.126183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4357f-46c1-488c-b1ff-0daa60060e0d service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:45.126183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4357f-46c1-488c-b1ff-0daa60060e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:45.126183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4357f-46c1-488c-b1ff-0daa60060e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:45.285374 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 872/1770] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:12:45.289681 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fa4b5-cb93-4053-a2c6-e55616b95366 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:45.289681 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-576fa4b5-cb93-4053-a2c6-e55616b95366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:45.389462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4357f-46c1-488c-b1ff-0daa60060e0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:45.397180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4357f-46c1-488c-b1ff-0daa60060e0d service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:45.441524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fa4b5-cb93-4053-a2c6-e55616b95366 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:45.446573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:45.447482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:45.447773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:45.448460 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] PATH_INFO: `/v3/users/3bb7ea35e1964544b538036ba4ed3457` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:45.449395 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bb7ea35e1964544b538036ba4ed3457)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:45.451511 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:45.457924 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 899/1771] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 775 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:12:45.458975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4357f-46c1-488c-b1ff-0daa60060e0d service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:45.473154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a1b4357f-46c1-488c-b1ff-0daa60060e0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:45.532857 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4357f-46c1-488c-b1ff-0daa60060e0d service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:45.547492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bb7ea35e1964544b538036ba4ed3457'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:45.550510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bb7ea35e1964544b538036ba4ed3457'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:45.551374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bb7ea35e1964544b538036ba4ed3457'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:45.568463 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bb7ea35e1964544b538036ba4ed3457'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:45.569849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:45.576827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:45.577781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:45.579161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] PATH_INFO: `/v3/projects/d8085f77d77f41249c68369b093d68cc` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:45.582936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8085f77d77f41249c68369b093d68cc)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:45.599197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:45.604770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bb7ea35e1964544b538036ba4ed3457'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:45.654334 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 873/1772] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:45 2022] DELETE /identity/v3/users/3bb7ea35e1964544b538036ba4ed3457 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:45.658704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:45.662944 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:45.740133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8085f77d77f41249c68369b093d68cc', 'request_id': 'req-fdb2f267-d1bc-43d8-96a0-744b71d059ef'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:45.766768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8085f77d77f41249c68369b093d68cc', 'request_id': 'req-fdb2f267-d1bc-43d8-96a0-744b71d059ef'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:45.777633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8085f77d77f41249c68369b093d68cc', 'request_id': 'req-fdb2f267-d1bc-43d8-96a0-744b71d059ef'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:45.812459 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:45.823324 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] The token cache is being invalidate because project d8085f77d77f41249c68369b093d68cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:45.823757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:45.827442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-fdb2f267-d1bc-43d8-96a0-744b71d059ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:45.830450 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 900/1773] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:45 2022] DELETE /identity/v3/projects/d8085f77d77f41249c68369b093d68cc => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:45.831802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626fdab-203b-47f5-8347-ce3e8aca27c1 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:45.832091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626fdab-203b-47f5-8347-ce3e8aca27c1 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:45.832368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626fdab-203b-47f5-8347-ce3e8aca27c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:45.883356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-0626fdab-203b-47f5-8347-ce3e8aca27c1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:45.884641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0626fdab-203b-47f5-8347-ce3e8aca27c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:45.895658 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484efca-69ce-4f3f-8315-2000e6e02763 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:45.919146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:46.040161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabab20c-6947-469e-a691-9f488c6cdcf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:46.040878 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabab20c-6947-469e-a691-9f488c6cdcf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:46.046319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabab20c-6947-469e-a691-9f488c6cdcf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:46.046752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabab20c-6947-469e-a691-9f488c6cdcf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:46.059913 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cabab20c-6947-469e-a691-9f488c6cdcf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:46.193384 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 901/1774] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:46.194459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626fdab-203b-47f5-8347-ce3e8aca27c1 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:46.196878 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0626fdab-203b-47f5-8347-ce3e8aca27c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:46.242200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626fdab-203b-47f5-8347-ce3e8aca27c1 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:46.262875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc7e62-a959-4941-9364-bb368349b3df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:46.263533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc7e62-a959-4941-9364-bb368349b3df service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:46.263821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc7e62-a959-4941-9364-bb368349b3df service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:46.264100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc7e62-a959-4941-9364-bb368349b3df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:46.270027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-22cc7e62-a959-4941-9364-bb368349b3df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:46.361261 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabab20c-6947-469e-a691-9f488c6cdcf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:46.362972 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabab20c-6947-469e-a691-9f488c6cdcf2 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:46.407039 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 874/1775] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 752 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:46.407722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabab20c-6947-469e-a691-9f488c6cdcf2 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:46.411877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cabab20c-6947-469e-a691-9f488c6cdcf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:46.444634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabab20c-6947-469e-a691-9f488c6cdcf2 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:46.471558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca0735-34b8-4913-840c-f556507bcba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:46.472747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca0735-34b8-4913-840c-f556507bcba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:46.473640 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca0735-34b8-4913-840c-f556507bcba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:46.474017 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca0735-34b8-4913-840c-f556507bcba0 admin admin] PATH_INFO: `/v3/users/70178178a21e43faaf3170fda862ca34` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:46.477597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca0735-34b8-4913-840c-f556507bcba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70178178a21e43faaf3170fda862ca34)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:46.484147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc7e62-a959-4941-9364-bb368349b3df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:46.484385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca0735-34b8-4913-840c-f556507bcba0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:46.487400 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc7e62-a959-4941-9364-bb368349b3df service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:46.563218 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 902/1776] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:46.567194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc7e62-a959-4941-9364-bb368349b3df service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:46.573415 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-22cc7e62-a959-4941-9364-bb368349b3df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:46.781769 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1eca0735-34b8-4913-840c-f556507bcba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70178178a21e43faaf3170fda862ca34'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:46.786896 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1eca0735-34b8-4913-840c-f556507bcba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70178178a21e43faaf3170fda862ca34'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:46.790502 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1eca0735-34b8-4913-840c-f556507bcba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70178178a21e43faaf3170fda862ca34'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:46.818963 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1eca0735-34b8-4913-840c-f556507bcba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70178178a21e43faaf3170fda862ca34'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:46.869769 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1eca0735-34b8-4913-840c-f556507bcba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70178178a21e43faaf3170fda862ca34'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:46.889122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc7e62-a959-4941-9364-bb368349b3df service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:46.902172 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 875/1777] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:46 2022] DELETE /identity/v3/users/70178178a21e43faaf3170fda862ca34 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:46.904667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab932971-a053-45ad-9486-139fd3535034 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:46.905302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab932971-a053-45ad-9486-139fd3535034 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:46.909027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e014e27-2494-4e7f-beca-580cd0b36d7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:46.909978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e014e27-2494-4e7f-beca-580cd0b36d7a service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:46.910452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e014e27-2494-4e7f-beca-580cd0b36d7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:46.910888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab932971-a053-45ad-9486-139fd3535034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:46.913127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e014e27-2494-4e7f-beca-580cd0b36d7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:46.916868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0e014e27-2494-4e7f-beca-580cd0b36d7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:46.930274 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:46.947450 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e014e27-2494-4e7f-beca-580cd0b36d7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:46.951922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e014e27-2494-4e7f-beca-580cd0b36d7a service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:46.963893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ab932971-a053-45ad-9486-139fd3535034 None None] MFA Rules not processed for user `e0865f21da0a4c95a84c0099a1b9a78b`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:46.965971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ab932971-a053-45ad-9486-139fd3535034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:46.980289 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 903/1778] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:46.983047 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e014e27-2494-4e7f-beca-580cd0b36d7a service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:46.985997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0e014e27-2494-4e7f-beca-580cd0b36d7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:47.040186 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e014e27-2494-4e7f-beca-580cd0b36d7a service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:47.152521 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 876/1779] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Jul 5 10:12:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:47.200247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b6b67-03fb-4357-8c70-a54336ac44bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:47.201586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b6b67-03fb-4357-8c70-a54336ac44bd service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:47.201586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b6b67-03fb-4357-8c70-a54336ac44bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:47.201835 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b6b67-03fb-4357-8c70-a54336ac44bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:47.227597 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-769b6b67-03fb-4357-8c70-a54336ac44bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:47.235750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:47.424024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b6b67-03fb-4357-8c70-a54336ac44bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:47.440595 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b6b67-03fb-4357-8c70-a54336ac44bd service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:47.462944 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab932971-a053-45ad-9486-139fd3535034 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:47.464840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ab932971-a053-45ad-9486-139fd3535034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:47.494935 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 904/1780] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:47.497192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab932971-a053-45ad-9486-139fd3535034 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:47.514695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9534f-1c90-420f-b394-dd93e42d6e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:47.517923 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9534f-1c90-420f-b394-dd93e42d6e18 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:47.518736 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9534f-1c90-420f-b394-dd93e42d6e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:47.519506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9534f-1c90-420f-b394-dd93e42d6e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:47.555993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9534f-1c90-420f-b394-dd93e42d6e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:47.557622 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9534f-1c90-420f-b394-dd93e42d6e18 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:47.592137 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 877/1781] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:12:49.571129 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76b381-d19c-4d0e-880c-6db295d859f4 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:49.573107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76b381-d19c-4d0e-880c-6db295d859f4 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:49.573574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76b381-d19c-4d0e-880c-6db295d859f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:49.641348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-9f76b381-d19c-4d0e-880c-6db295d859f4 None None] MFA Rules not processed for user `51e20b9a489342d2b8f0f85542ab370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:49.643999 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9f76b381-d19c-4d0e-880c-6db295d859f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:49.910452 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 905/1782] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:50.756714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9534f-1c90-420f-b394-dd93e42d6e18 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:50.758896 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f1d9534f-1c90-420f-b394-dd93e42d6e18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:51.003958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9534f-1c90-420f-b394-dd93e42d6e18 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:51.033255 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:51.034588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:51.035035 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:51.035554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] PATH_INFO: `/v3/projects/afeb30b15e134306a3dafd0f00f63c9e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:51.037297 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afeb30b15e134306a3dafd0f00f63c9e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:51.039893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:51.161734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afeb30b15e134306a3dafd0f00f63c9e', 'request_id': 'req-1395e6bf-facf-4a66-802f-72f5308a3671'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.177047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afeb30b15e134306a3dafd0f00f63c9e', 'request_id': 'req-1395e6bf-facf-4a66-802f-72f5308a3671'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.190021 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afeb30b15e134306a3dafd0f00f63c9e', 'request_id': 'req-1395e6bf-facf-4a66-802f-72f5308a3671'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.216850 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] The token cache is being invalidate because project afeb30b15e134306a3dafd0f00f63c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:51.217337 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.219888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.238349 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 878/1783] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:50 2022] DELETE /identity/v3/projects/afeb30b15e134306a3dafd0f00f63c9e => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:51.255475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1dbc2d-48ba-4ac3-aab1-a3d254c95244 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:51.255981 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1dbc2d-48ba-4ac3-aab1-a3d254c95244 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:51.256477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1dbc2d-48ba-4ac3-aab1-a3d254c95244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:51.298525 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:51.300397 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:51.322681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-1d1dbc2d-48ba-4ac3-aab1-a3d254c95244 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:51.331173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1d1dbc2d-48ba-4ac3-aab1-a3d254c95244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:51.390332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:51.508410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395e6bf-facf-4a66-802f-72f5308a3671 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:51.527720 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea700317-e866-405c-bffd-72b577b0fec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:51.531341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea700317-e866-405c-bffd-72b577b0fec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:51.536051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea700317-e866-405c-bffd-72b577b0fec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:51.536898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea700317-e866-405c-bffd-72b577b0fec5 admin admin] PATH_INFO: `/v3/users/47f0474641244fbfbac0fed8c83b8e1b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:51.541531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea700317-e866-405c-bffd-72b577b0fec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f0474641244fbfbac0fed8c83b8e1b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:51.548483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea700317-e866-405c-bffd-72b577b0fec5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:51.565771 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 906/1784] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:51.575889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1dbc2d-48ba-4ac3-aab1-a3d254c95244 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:51.585997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1d1dbc2d-48ba-4ac3-aab1-a3d254c95244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:51.614504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1dbc2d-48ba-4ac3-aab1-a3d254c95244 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:51.625639 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:51.626943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:51.627399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:51.627850 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] PATH_INFO: `/v3/projects/aa91abd5cadb4bd9b7ace631450a8d1b` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:51.634511 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa91abd5cadb4bd9b7ace631450a8d1b)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:51.637735 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:51.671614 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ea700317-e866-405c-bffd-72b577b0fec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f0474641244fbfbac0fed8c83b8e1b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.673433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ea700317-e866-405c-bffd-72b577b0fec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f0474641244fbfbac0fed8c83b8e1b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.674707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ea700317-e866-405c-bffd-72b577b0fec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f0474641244fbfbac0fed8c83b8e1b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.705742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ea700317-e866-405c-bffd-72b577b0fec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f0474641244fbfbac0fed8c83b8e1b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.725733 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ea700317-e866-405c-bffd-72b577b0fec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f0474641244fbfbac0fed8c83b8e1b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.795729 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 879/1785] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:51 2022] DELETE /identity/v3/users/47f0474641244fbfbac0fed8c83b8e1b => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:51.796821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14350c9f-bede-401f-bdea-407e2e9d31ef None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:51.799181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:51.801318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14350c9f-bede-401f-bdea-407e2e9d31ef None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:51.802303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14350c9f-bede-401f-bdea-407e2e9d31ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:51.806136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa91abd5cadb4bd9b7ace631450a8d1b', 'request_id': 'req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.827473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa91abd5cadb4bd9b7ace631450a8d1b', 'request_id': 'req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.856727 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa91abd5cadb4bd9b7ace631450a8d1b', 'request_id': 'req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.871113 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-14350c9f-bede-401f-bdea-407e2e9d31ef None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:51.880492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-14350c9f-bede-401f-bdea-407e2e9d31ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:51.898309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] The token cache is being invalidate because project aa91abd5cadb4bd9b7ace631450a8d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:51.899064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.902339 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-7e1fa5eb-655c-40a1-a4b5-0f044da5198d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:51.936580 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 907/1786] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:51 2022] DELETE /identity/v3/projects/aa91abd5cadb4bd9b7ace631450a8d1b => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:51.942832 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17136d65-b975-48e8-a65a-37dfc89e880d None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:51.943131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17136d65-b975-48e8-a65a-37dfc89e880d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:51.946829 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17136d65-b975-48e8-a65a-37dfc89e880d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:52.028442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-17136d65-b975-48e8-a65a-37dfc89e880d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:52.032654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-17136d65-b975-48e8-a65a-37dfc89e880d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:52.205033 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 880/1787] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:52.211720 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e4d58-2dcc-4e9d-be24-674099413a39 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:52.219006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e4d58-2dcc-4e9d-be24-674099413a39 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:52.220753 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e4d58-2dcc-4e9d-be24-674099413a39 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:52.222638 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 881/1788] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:12:52 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:12:52.230687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e4d58-2dcc-4e9d-be24-674099413a39 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:52.234686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bb5e4d58-2dcc-4e9d-be24-674099413a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:52.263467 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 908/1789] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:52.264389 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17136d65-b975-48e8-a65a-37dfc89e880d None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:52.299327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e4d58-2dcc-4e9d-be24-674099413a39 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:52.307725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17136d65-b975-48e8-a65a-37dfc89e880d None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:52.314720 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19659f-eda4-438d-aa5d-020b2ed71018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:52.315803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19659f-eda4-438d-aa5d-020b2ed71018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:52.316200 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19659f-eda4-438d-aa5d-020b2ed71018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:52.316624 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19659f-eda4-438d-aa5d-020b2ed71018 admin admin] PATH_INFO: `/v3/users/cc43aa71d0294f9aa31daace8053a93e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:52.334470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19659f-eda4-438d-aa5d-020b2ed71018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc43aa71d0294f9aa31daace8053a93e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:52.337299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19659f-eda4-438d-aa5d-020b2ed71018 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:52.342984 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72771ff-b31c-41d3-8732-83b27e8d4e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:52.349503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72771ff-b31c-41d3-8732-83b27e8d4e99 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:52.349743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72771ff-b31c-41d3-8732-83b27e8d4e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:52.350024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72771ff-b31c-41d3-8732-83b27e8d4e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:52.373177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72771ff-b31c-41d3-8732-83b27e8d4e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:52.373177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72771ff-b31c-41d3-8732-83b27e8d4e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:52.382724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72771ff-b31c-41d3-8732-83b27e8d4e99 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:52.475549 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 909/1790] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:52.490256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:52.490534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72771ff-b31c-41d3-8732-83b27e8d4e99 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:52.499090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0f19659f-eda4-438d-aa5d-020b2ed71018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc43aa71d0294f9aa31daace8053a93e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:52.501231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0f19659f-eda4-438d-aa5d-020b2ed71018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc43aa71d0294f9aa31daace8053a93e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:52.503502 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0f19659f-eda4-438d-aa5d-020b2ed71018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc43aa71d0294f9aa31daace8053a93e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:52.503793 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f72771ff-b31c-41d3-8732-83b27e8d4e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:52.535656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0f19659f-eda4-438d-aa5d-020b2ed71018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc43aa71d0294f9aa31daace8053a93e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:52.583678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0f19659f-eda4-438d-aa5d-020b2ed71018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc43aa71d0294f9aa31daace8053a93e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:52.620465 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 882/1791] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:52 2022] DELETE /identity/v3/users/cc43aa71d0294f9aa31daace8053a93e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:52.624927 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fade5-7fb5-4c0a-ad72-b431b198545e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:52.626115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fade5-7fb5-4c0a-ad72-b431b198545e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:52.626570 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fade5-7fb5-4c0a-ad72-b431b198545e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:52.709775 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fc5fade5-7fb5-4c0a-ad72-b431b198545e None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:52.722185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fc5fade5-7fb5-4c0a-ad72-b431b198545e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:52.779604 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72771ff-b31c-41d3-8732-83b27e8d4e99 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:53.004618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b9e07-bf16-4df9-aa7d-98fbd6564c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:53.008423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b9e07-bf16-4df9-aa7d-98fbd6564c62 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:53.009427 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 883/1792] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:12:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:53.010470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fade5-7fb5-4c0a-ad72-b431b198545e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:53.013089 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b9e07-bf16-4df9-aa7d-98fbd6564c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:53.014442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b9e07-bf16-4df9-aa7d-98fbd6564c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:53.022309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-820b9e07-bf16-4df9-aa7d-98fbd6564c62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:53.027347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fc5fade5-7fb5-4c0a-ad72-b431b198545e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:53.188471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:53.278167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b9e07-bf16-4df9-aa7d-98fbd6564c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:53.280969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b9e07-bf16-4df9-aa7d-98fbd6564c62 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:53.313396 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 910/1793] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 826 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:53.364760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fade5-7fb5-4c0a-ad72-b431b198545e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:53.389912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685b816-d579-4deb-9621-d843ba024ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:53.392750 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685b816-d579-4deb-9621-d843ba024ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:53.393788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685b816-d579-4deb-9621-d843ba024ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:53.395596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685b816-d579-4deb-9621-d843ba024ad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:53.396640 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685b816-d579-4deb-9621-d843ba024ad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:53.407236 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685b816-d579-4deb-9621-d843ba024ad5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:53.539546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b9e07-bf16-4df9-aa7d-98fbd6564c62 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:53.540354 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 884/1794] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:53 2022] POST /identity/v3/projects => generated 379 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:53.559590 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b9e07-bf16-4df9-aa7d-98fbd6564c62 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:53.816746 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09534af7-7bbd-4a9e-b4d9-e264e201d32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:53.817783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09534af7-7bbd-4a9e-b4d9-e264e201d32c admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:53.818075 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09534af7-7bbd-4a9e-b4d9-e264e201d32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:53.818440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09534af7-7bbd-4a9e-b4d9-e264e201d32c admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:53.819230 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09534af7-7bbd-4a9e-b4d9-e264e201d32c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:53.822092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09534af7-7bbd-4a9e-b4d9-e264e201d32c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:53.856342 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 911/1795] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:53 2022] POST /identity/v3/users => generated 290 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:53.856807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685b816-d579-4deb-9621-d843ba024ad5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:53.874689 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685b816-d579-4deb-9621-d843ba024ad5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:53.897673 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636c5a9-47a8-4c5d-9350-de9dc8cea465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:53.901380 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636c5a9-47a8-4c5d-9350-de9dc8cea465 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:53.901765 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636c5a9-47a8-4c5d-9350-de9dc8cea465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:53.903843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636c5a9-47a8-4c5d-9350-de9dc8cea465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:53.905992 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636c5a9-47a8-4c5d-9350-de9dc8cea465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:53.911156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636c5a9-47a8-4c5d-9350-de9dc8cea465 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:53.956084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09534af7-7bbd-4a9e-b4d9-e264e201d32c admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:53.963034 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-09534af7-7bbd-4a9e-b4d9-e264e201d32c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:53.968340 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 885/1796] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:53.972690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636c5a9-47a8-4c5d-9350-de9dc8cea465 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:54.002351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636c5a9-47a8-4c5d-9350-de9dc8cea465 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:54.019710 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d82837-ae9b-4561-bdcc-b171d7d95db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:54.020906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d82837-ae9b-4561-bdcc-b171d7d95db0 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:54.021523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d82837-ae9b-4561-bdcc-b171d7d95db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:54.021909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d82837-ae9b-4561-bdcc-b171d7d95db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:54.023352 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d82837-ae9b-4561-bdcc-b171d7d95db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:54.026169 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d82837-ae9b-4561-bdcc-b171d7d95db0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:54.040445 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 886/1797] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:54.044600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d82837-ae9b-4561-bdcc-b171d7d95db0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:54.093189 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d82837-ae9b-4561-bdcc-b171d7d95db0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:54.111138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6620df-e792-4374-9e7d-103facb8d6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:54.114442 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6620df-e792-4374-9e7d-103facb8d6ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:54.114948 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6620df-e792-4374-9e7d-103facb8d6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:54.115423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6620df-e792-4374-9e7d-103facb8d6ce admin admin] PATH_INFO: `/v3/projects/021fea6560f949b082660d458130c7b5/users/654a9b1ae9c147a09d890a879f5d95f5/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:54.187498 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6620df-e792-4374-9e7d-103facb8d6ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=021fea6560f949b082660d458130c7b5, user_id=654a9b1ae9c147a09d890a879f5d95f5, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:54.191018 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6620df-e792-4374-9e7d-103facb8d6ce admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:54.199395 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:54.204957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09534af7-7bbd-4a9e-b4d9-e264e201d32c admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:54.215424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da37e43-3f65-487d-b0d9-70ba6735e3f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:54.216501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da37e43-3f65-487d-b0d9-70ba6735e3f6 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:54.216887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da37e43-3f65-487d-b0d9-70ba6735e3f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:54.217437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da37e43-3f65-487d-b0d9-70ba6735e3f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:54.232940 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 887/1798] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:54 2022] PUT /identity/v3/projects/021fea6560f949b082660d458130c7b5/users/654a9b1ae9c147a09d890a879f5d95f5/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:54.233187 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da37e43-3f65-487d-b0d9-70ba6735e3f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:54.235109 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91c717-333d-4e7b-8999-9530182c39c5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:54.235461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91c717-333d-4e7b-8999-9530182c39c5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:54.235834 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91c717-333d-4e7b-8999-9530182c39c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:54.235981 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da37e43-3f65-487d-b0d9-70ba6735e3f6 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:54.257374 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 912/1799] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:54.315054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8d91c717-333d-4e7b-8999-9530182c39c5 None None] MFA Rules not processed for user `654a9b1ae9c147a09d890a879f5d95f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:54.316381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8d91c717-333d-4e7b-8999-9530182c39c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:54.500086 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 888/1800] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:54.501473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da37e43-3f65-487d-b0d9-70ba6735e3f6 service nova] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:54.516395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da37e43-3f65-487d-b0d9-70ba6735e3f6 service nova] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:54.673340 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:54.727490 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f77b5-6705-4b45-bed8-a99029f5854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:54.728034 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f77b5-6705-4b45-bed8-a99029f5854f admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:54.728483 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f77b5-6705-4b45-bed8-a99029f5854f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:54.729755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f77b5-6705-4b45-bed8-a99029f5854f admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:54.731125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f77b5-6705-4b45-bed8-a99029f5854f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:54.738560 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f77b5-6705-4b45-bed8-a99029f5854f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:54.802844 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 913/1801] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:54 2022] POST /identity/v3/projects => generated 379 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:54.804534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91c717-333d-4e7b-8999-9530182c39c5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:54.854729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91c717-333d-4e7b-8999-9530182c39c5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:55.080868 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e115194-bc4b-48e2-9aec-e591625f6b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:55.082344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e115194-bc4b-48e2-9aec-e591625f6b8d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:55.082698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e115194-bc4b-48e2-9aec-e591625f6b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:55.082984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e115194-bc4b-48e2-9aec-e591625f6b8d admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:55.087946 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e115194-bc4b-48e2-9aec-e591625f6b8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:55.134627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e115194-bc4b-48e2-9aec-e591625f6b8d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:55.175172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f77b5-6705-4b45-bed8-a99029f5854f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:55.177492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a22f77b5-6705-4b45-bed8-a99029f5854f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:55.217116 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 889/1802] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:54 2022] POST /identity/v3/users => generated 296 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:55.221758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e115194-bc4b-48e2-9aec-e591625f6b8d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:55.224347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f77b5-6705-4b45-bed8-a99029f5854f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:55.240517 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:55.241422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:55.241710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:55.241995 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] PATH_INFO: `/v3/projects/751e004e76d74415a25f2c5cb0969516` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:55.250038 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e115194-bc4b-48e2-9aec-e591625f6b8d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:55.252731 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=751e004e76d74415a25f2c5cb0969516)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:55.259114 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:55.272483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8c458-0ac4-4e57-9d49-d38724149e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:55.273406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8c458-0ac4-4e57-9d49-d38724149e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:55.273756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8c458-0ac4-4e57-9d49-d38724149e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:55.274184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8c458-0ac4-4e57-9d49-d38724149e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:55.275095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8c458-0ac4-4e57-9d49-d38724149e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:55.277053 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8c458-0ac4-4e57-9d49-d38724149e8c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:55.304214 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 890/1803] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:55.306597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8c458-0ac4-4e57-9d49-d38724149e8c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:55.328120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8c458-0ac4-4e57-9d49-d38724149e8c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:55.345153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc67acc-8023-402e-afff-d3180359e50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:55.348115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc67acc-8023-402e-afff-d3180359e50a admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:55.349437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc67acc-8023-402e-afff-d3180359e50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:55.350858 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc67acc-8023-402e-afff-d3180359e50a admin admin] PATH_INFO: `/v3/projects/247c43ecafa346658e71319e1e2589a5/users/07a932c0c8c742eea2200a87320c50a4/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:55.379809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '751e004e76d74415a25f2c5cb0969516', 'request_id': 'req-f573c397-5453-4277-b7f0-23abeb730aac'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:55.399854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc67acc-8023-402e-afff-d3180359e50a admin admin] RBAC: Authorizing `identity:create_grant(project_id=247c43ecafa346658e71319e1e2589a5, user_id=07a932c0c8c742eea2200a87320c50a4, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:55.404731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc67acc-8023-402e-afff-d3180359e50a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:55.406206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '751e004e76d74415a25f2c5cb0969516', 'request_id': 'req-f573c397-5453-4277-b7f0-23abeb730aac'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:55.420720 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '751e004e76d74415a25f2c5cb0969516', 'request_id': 'req-f573c397-5453-4277-b7f0-23abeb730aac'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:55.437210 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 891/1804] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:55 2022] PUT /identity/v3/projects/247c43ecafa346658e71319e1e2589a5/users/07a932c0c8c742eea2200a87320c50a4/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:55.440501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc67acc-8023-402e-afff-d3180359e50a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:55.445492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cdc67acc-8023-402e-afff-d3180359e50a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:55.451359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] The token cache is being invalidate because project 751e004e76d74415a25f2c5cb0969516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:55.452931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:55.456218 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:55.463653 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 914/1805] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:55 2022] DELETE /identity/v3/projects/751e004e76d74415a25f2c5cb0969516 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:55.464886 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:55.468103 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:55.611096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:55.684469 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc67acc-8023-402e-afff-d3180359e50a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:55.688496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573c397-5453-4277-b7f0-23abeb730aac admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:55.700147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:55.702687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:55.703694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a693a-dd7a-48d7-9c94-baa49e2aa251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:55.704414 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:55.704858 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] PATH_INFO: `/v3/projects/0b91f9ac0a1443e2a7fc986e9464caa4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:55.707052 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a693a-dd7a-48d7-9c94-baa49e2aa251 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:55.708315 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b91f9ac0a1443e2a7fc986e9464caa4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:55.708673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a693a-dd7a-48d7-9c94-baa49e2aa251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:55.709053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a693a-dd7a-48d7-9c94-baa49e2aa251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:55.711122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a693a-dd7a-48d7-9c94-baa49e2aa251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:55.713910 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:55.715439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a693a-dd7a-48d7-9c94-baa49e2aa251 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:55.738698 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 915/1806] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:55.741401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca24e-0cbe-46fa-91a4-be6efa3f30a4 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:55.743571 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca24e-0cbe-46fa-91a4-be6efa3f30a4 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:55.745148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca24e-0cbe-46fa-91a4-be6efa3f30a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:55.803500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a1fca24e-0cbe-46fa-91a4-be6efa3f30a4 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:55.804990 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a1fca24e-0cbe-46fa-91a4-be6efa3f30a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:55.841762 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b91f9ac0a1443e2a7fc986e9464caa4', 'request_id': 'req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:55.861432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b91f9ac0a1443e2a7fc986e9464caa4', 'request_id': 'req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:55.873103 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b91f9ac0a1443e2a7fc986e9464caa4', 'request_id': 'req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:55.906347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] The token cache is being invalidate because project 0b91f9ac0a1443e2a7fc986e9464caa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:55.906648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:55.912117 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:55.914406 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 892/1807] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:55 2022] DELETE /identity/v3/projects/0b91f9ac0a1443e2a7fc986e9464caa4 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:55.917747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:55.919472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:55.934683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:56.007544 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 916/1808] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:56.007544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fca24e-0cbe-46fa-91a4-be6efa3f30a4 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:56.013817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a1fca24e-0cbe-46fa-91a4-be6efa3f30a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:56.036535 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fca24e-0cbe-46fa-91a4-be6efa3f30a4 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:56.050706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f36e2-350a-4fbf-89c9-a6603f76a83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:56.051469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f36e2-350a-4fbf-89c9-a6603f76a83e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.053106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f36e2-350a-4fbf-89c9-a6603f76a83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.053405 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f36e2-350a-4fbf-89c9-a6603f76a83e admin admin] PATH_INFO: `/v3/users/77954d74b22244a3b8120402187cd14d` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.054994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f36e2-350a-4fbf-89c9-a6603f76a83e admin admin] RBAC: Authorizing `identity:delete_user(user_id=77954d74b22244a3b8120402187cd14d)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:56.079851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f36e2-350a-4fbf-89c9-a6603f76a83e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:56.102029 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982dd6c1-1c45-46fa-baa3-6be5e28ba6ab admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:56.114098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b6609-7f24-43a3-affe-8e077d027dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:56.116900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b6609-7f24-43a3-affe-8e077d027dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.117202 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b6609-7f24-43a3-affe-8e077d027dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.117488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b6609-7f24-43a3-affe-8e077d027dc2 admin admin] PATH_INFO: `/v3/projects/247c43ecafa346658e71319e1e2589a5/users/07a932c0c8c742eea2200a87320c50a4/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.120174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b6609-7f24-43a3-affe-8e077d027dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=247c43ecafa346658e71319e1e2589a5, user_id=07a932c0c8c742eea2200a87320c50a4, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:56.124701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b6609-7f24-43a3-affe-8e077d027dc2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:56.150785 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 893/1809] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:55 2022] PUT /identity/v3/projects/247c43ecafa346658e71319e1e2589a5/users/07a932c0c8c742eea2200a87320c50a4/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:56.150785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d4618-b3b1-4f38-81d5-b491a2e224c1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.150785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d4618-b3b1-4f38-81d5-b491a2e224c1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.150785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d4618-b3b1-4f38-81d5-b491a2e224c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.185288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ff7d4618-b3b1-4f38-81d5-b491a2e224c1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:56.187429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ff7d4618-b3b1-4f38-81d5-b491a2e224c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:56.195491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4f4f36e2-350a-4fbf-89c9-a6603f76a83e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77954d74b22244a3b8120402187cd14d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:56.197373 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4f4f36e2-350a-4fbf-89c9-a6603f76a83e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77954d74b22244a3b8120402187cd14d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:56.232274 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4f4f36e2-350a-4fbf-89c9-a6603f76a83e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77954d74b22244a3b8120402187cd14d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:56.241693 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4f4f36e2-350a-4fbf-89c9-a6603f76a83e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77954d74b22244a3b8120402187cd14d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:56.244251 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4f4f36e2-350a-4fbf-89c9-a6603f76a83e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77954d74b22244a3b8120402187cd14d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:56.270762 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 917/1810] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:56 2022] DELETE /identity/v3/users/77954d74b22244a3b8120402187cd14d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:56.274713 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d432a6-3ef3-450f-a850-a5ee93238972 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.275010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d432a6-3ef3-450f-a850-a5ee93238972 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.276147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d432a6-3ef3-450f-a850-a5ee93238972 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.282107 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 894/1811] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:56.283764 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76260cc-42de-4eb4-8802-4b6686c3406c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.283976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76260cc-42de-4eb4-8802-4b6686c3406c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.284260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76260cc-42de-4eb4-8802-4b6686c3406c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.285493 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 918/1812] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:12:56 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:12:56.287937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d432a6-3ef3-450f-a850-a5ee93238972 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:56.291142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:56.319364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b76260cc-42de-4eb4-8802-4b6686c3406c None None] MFA Rules not processed for user `07a932c0c8c742eea2200a87320c50a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:56.320731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b76260cc-42de-4eb4-8802-4b6686c3406c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:56.404385 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d432a6-3ef3-450f-a850-a5ee93238972 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:56.413704 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 895/1813] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:56.413704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f81434-9363-4e23-9624-5d2d121a2edb None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.413989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0b378-cff9-4a3d-868c-f18a7d8a28da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:56.414093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f81434-9363-4e23-9624-5d2d121a2edb None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.414093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f81434-9363-4e23-9624-5d2d121a2edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.414300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0b378-cff9-4a3d-868c-f18a7d8a28da admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.414300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0b378-cff9-4a3d-868c-f18a7d8a28da admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.414300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0b378-cff9-4a3d-868c-f18a7d8a28da admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.414300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0b378-cff9-4a3d-868c-f18a7d8a28da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:56.414300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0b378-cff9-4a3d-868c-f18a7d8a28da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:56.416304 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0b378-cff9-4a3d-868c-f18a7d8a28da admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:56.433752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-79f81434-9363-4e23-9624-5d2d121a2edb None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:56.435253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-79f81434-9363-4e23-9624-5d2d121a2edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:56.437834 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 919/1814] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:56.439527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0b378-cff9-4a3d-868c-f18a7d8a28da admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:56.441353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-06e0b378-cff9-4a3d-868c-f18a7d8a28da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:56.535654 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 896/1815] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:12:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:56.538569 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f81434-9363-4e23-9624-5d2d121a2edb None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:56.547018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0b378-cff9-4a3d-868c-f18a7d8a28da admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:56.552017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1ce46-e33f-4b38-83a1-f3af3714e2cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:56.553388 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1ce46-e33f-4b38-83a1-f3af3714e2cc service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.554088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1ce46-e33f-4b38-83a1-f3af3714e2cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.554578 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1ce46-e33f-4b38-83a1-f3af3714e2cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.563150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f81434-9363-4e23-9624-5d2d121a2edb None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:56.567996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2cf7f-4322-49e9-97bf-c4b56b87e475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:56.568855 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2cf7f-4322-49e9-97bf-c4b56b87e475 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.569145 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2cf7f-4322-49e9-97bf-c4b56b87e475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.569415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2cf7f-4322-49e9-97bf-c4b56b87e475 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.570273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2cf7f-4322-49e9-97bf-c4b56b87e475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:56.572389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2cf7f-4322-49e9-97bf-c4b56b87e475 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:56.578840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1ce46-e33f-4b38-83a1-f3af3714e2cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:56.580751 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1ce46-e33f-4b38-83a1-f3af3714e2cc service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:56.586459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:56.602800 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 920/1816] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:56.607014 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 897/1817] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:56 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:56.611336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1ce46-e33f-4b38-83a1-f3af3714e2cc service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:56.627991 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1ce46-e33f-4b38-83a1-f3af3714e2cc service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:56.756380 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2cf7f-4322-49e9-97bf-c4b56b87e475 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:56.759117 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c9b2cf7f-4322-49e9-97bf-c4b56b87e475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:56.771982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334902be-a6b1-470b-b82a-175921c52939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:56.774565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334902be-a6b1-470b-b82a-175921c52939 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.774939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334902be-a6b1-470b-b82a-175921c52939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.775223 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334902be-a6b1-470b-b82a-175921c52939 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.777176 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334902be-a6b1-470b-b82a-175921c52939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:56.783465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334902be-a6b1-470b-b82a-175921c52939 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:56.834505 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 921/1818] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:56 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:56.839667 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334902be-a6b1-470b-b82a-175921c52939 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:56.863112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334902be-a6b1-470b-b82a-175921c52939 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:56.885091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50beae3-462e-4a5b-9366-11b7e34bea5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:56.886076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50beae3-462e-4a5b-9366-11b7e34bea5b admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.886076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50beae3-462e-4a5b-9366-11b7e34bea5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.886354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50beae3-462e-4a5b-9366-11b7e34bea5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.887178 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50beae3-462e-4a5b-9366-11b7e34bea5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:56.889232 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50beae3-462e-4a5b-9366-11b7e34bea5b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:56.904600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:56.916043 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 922/1819] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:56.920073 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50beae3-462e-4a5b-9366-11b7e34bea5b admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:56.938838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50beae3-462e-4a5b-9366-11b7e34bea5b admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:56.943034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2cf7f-4322-49e9-97bf-c4b56b87e475 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:56.950240 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add775e6-5bb4-4cf2-ae7c-d3215af4ab45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:56.950874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add775e6-5bb4-4cf2-ae7c-d3215af4ab45 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.951165 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add775e6-5bb4-4cf2-ae7c-d3215af4ab45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.951446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add775e6-5bb4-4cf2-ae7c-d3215af4ab45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.953482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add775e6-5bb4-4cf2-ae7c-d3215af4ab45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:56.955149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add775e6-5bb4-4cf2-ae7c-d3215af4ab45 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:56.957899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b59ca9-4e67-4ed6-a9eb-08e89977712b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:56.958614 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b59ca9-4e67-4ed6-a9eb-08e89977712b service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:56.958898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b59ca9-4e67-4ed6-a9eb-08e89977712b service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:56.959338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b59ca9-4e67-4ed6-a9eb-08e89977712b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:56.976866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b59ca9-4e67-4ed6-a9eb-08e89977712b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:56.978930 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b59ca9-4e67-4ed6-a9eb-08e89977712b service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:56.986179 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 923/1820] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:56.986858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add775e6-5bb4-4cf2-ae7c-d3215af4ab45 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:56.991678 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-add775e6-5bb4-4cf2-ae7c-d3215af4ab45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:57.001142 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 898/1821] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:57.013254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b59ca9-4e67-4ed6-a9eb-08e89977712b service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:57.020508 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add775e6-5bb4-4cf2-ae7c-d3215af4ab45 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:57.032720 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b85547-f178-4e9c-b518-9989c4ce3aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:57.033647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b85547-f178-4e9c-b518-9989c4ce3aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.034065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b85547-f178-4e9c-b518-9989c4ce3aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.034376 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b85547-f178-4e9c-b518-9989c4ce3aaf admin admin] PATH_INFO: `/v3/users/017ad6c76196440fa95f2e38f65d0e26` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.036312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b85547-f178-4e9c-b518-9989c4ce3aaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=017ad6c76196440fa95f2e38f65d0e26)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:57.038259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b85547-f178-4e9c-b518-9989c4ce3aaf admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:57.041178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b59ca9-4e67-4ed6-a9eb-08e89977712b service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:57.049399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e81c3e-57f4-41a8-b687-237eb2f83dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:57.050477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e81c3e-57f4-41a8-b687-237eb2f83dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.050767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e81c3e-57f4-41a8-b687-237eb2f83dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.051046 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e81c3e-57f4-41a8-b687-237eb2f83dd3 admin admin] PATH_INFO: `/v3/projects/ff6dcf0f79f3441895c662a922c80896/users/68d5f03462754e4e91328dfb84971c90/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.087330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e81c3e-57f4-41a8-b687-237eb2f83dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6dcf0f79f3441895c662a922c80896, user_id=68d5f03462754e4e91328dfb84971c90, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:57.090454 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e81c3e-57f4-41a8-b687-237eb2f83dd3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:57.109329 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 899/1822] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:57 2022] PUT /identity/v3/projects/ff6dcf0f79f3441895c662a922c80896/users/68d5f03462754e4e91328dfb84971c90/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:57.110279 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e81c3e-57f4-41a8-b687-237eb2f83dd3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:57.117974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-21b85547-f178-4e9c-b518-9989c4ce3aaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '017ad6c76196440fa95f2e38f65d0e26'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.118770 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-21b85547-f178-4e9c-b518-9989c4ce3aaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '017ad6c76196440fa95f2e38f65d0e26'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.120383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e6e81c3e-57f4-41a8-b687-237eb2f83dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:57.135895 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-21b85547-f178-4e9c-b518-9989c4ce3aaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '017ad6c76196440fa95f2e38f65d0e26'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.146088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-21b85547-f178-4e9c-b518-9989c4ce3aaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '017ad6c76196440fa95f2e38f65d0e26'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.147333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-21b85547-f178-4e9c-b518-9989c4ce3aaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '017ad6c76196440fa95f2e38f65d0e26'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.165984 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 924/1823] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:56 2022] DELETE /identity/v3/users/017ad6c76196440fa95f2e38f65d0e26 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:57.168064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c97f32-d63c-404b-8c20-ea779b909d38 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.168301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c97f32-d63c-404b-8c20-ea779b909d38 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.169131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c97f32-d63c-404b-8c20-ea779b909d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.199543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-d7c97f32-d63c-404b-8c20-ea779b909d38 None None] MFA Rules not processed for user `68d5f03462754e4e91328dfb84971c90`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:57.200416 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d7c97f32-d63c-404b-8c20-ea779b909d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:57.202521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e81c3e-57f4-41a8-b687-237eb2f83dd3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:57.229944 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:57.268241 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:57.271077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.274829 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.275354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] PATH_INFO: `/v3/projects/eaa23b69daa541caa8d65f4fa976d0ef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.276671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa23b69daa541caa8d65f4fa976d0ef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:57.278956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:57.290598 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 925/1824] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:57.291921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24edf-4d43-4947-aae1-d4adb31f71dd None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.292214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24edf-4d43-4947-aae1-d4adb31f71dd None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.292485 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24edf-4d43-4947-aae1-d4adb31f71dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.314123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-9be24edf-4d43-4947-aae1-d4adb31f71dd None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:57.315891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9be24edf-4d43-4947-aae1-d4adb31f71dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:57.334362 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 926/1825] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:57.337210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be24edf-4d43-4947-aae1-d4adb31f71dd None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:57.338746 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9be24edf-4d43-4947-aae1-d4adb31f71dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:57.348256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa23b69daa541caa8d65f4fa976d0ef', 'request_id': 'req-2267052b-cf30-4eca-8e7f-5498f70f4bc5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.359492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa23b69daa541caa8d65f4fa976d0ef', 'request_id': 'req-2267052b-cf30-4eca-8e7f-5498f70f4bc5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.369477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa23b69daa541caa8d65f4fa976d0ef', 'request_id': 'req-2267052b-cf30-4eca-8e7f-5498f70f4bc5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.382891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:57.400169 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] The token cache is being invalidate because project eaa23b69daa541caa8d65f4fa976d0ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:57.400670 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.401787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.406935 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 900/1826] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:57 2022] DELETE /identity/v3/projects/eaa23b69daa541caa8d65f4fa976d0ef => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:57.406935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:57.406935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:57.442106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be24edf-4d43-4947-aae1-d4adb31f71dd None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:57.486191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267052b-cf30-4eca-8e7f-5498f70f4bc5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:57.491390 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42acf97-7266-41b2-a569-f69d1be27c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:57.492071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42acf97-7266-41b2-a569-f69d1be27c6d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.492358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42acf97-7266-41b2-a569-f69d1be27c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.492639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42acf97-7266-41b2-a569-f69d1be27c6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.493377 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42acf97-7266-41b2-a569-f69d1be27c6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:57.495436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42acf97-7266-41b2-a569-f69d1be27c6d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:57.514431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f405cd-55eb-48ed-b49c-869ff269c85a tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a932c0c8c742eea2200a87320c50a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '247c43ecafa346658e71319e1e2589a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:57.515392 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f405cd-55eb-48ed-b49c-869ff269c85a tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.515731 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f405cd-55eb-48ed-b49c-869ff269c85a tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.516050 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f405cd-55eb-48ed-b49c-869ff269c85a tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.516819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f405cd-55eb-48ed-b49c-869ff269c85a tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:57.517132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f405cd-55eb-48ed-b49c-869ff269c85a tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:57.518935 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f405cd-55eb-48ed-b49c-869ff269c85a tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:57.526075 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 901/1827] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:57 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:57.526859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42acf97-7266-41b2-a569-f69d1be27c6d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:57.528555 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f42acf97-7266-41b2-a569-f69d1be27c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:57.534132 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 927/1828] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:57.535068 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f405cd-55eb-48ed-b49c-869ff269c85a tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:57.536846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-16f405cd-55eb-48ed-b49c-869ff269c85a tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:57.553768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f405cd-55eb-48ed-b49c-869ff269c85a tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:57.601644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42acf97-7266-41b2-a569-f69d1be27c6d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:57.611573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9accaf-b1cd-4355-ae2c-a300dc28939f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:57.612227 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9accaf-b1cd-4355-ae2c-a300dc28939f service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.612509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9accaf-b1cd-4355-ae2c-a300dc28939f service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.612782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9accaf-b1cd-4355-ae2c-a300dc28939f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.614631 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1d9accaf-b1cd-4355-ae2c-a300dc28939f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:57.622741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:57.623646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.623926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.624197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] PATH_INFO: `/v3/users/0796890e391a4ffeabb664e0e64c8b95` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.625441 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0796890e391a4ffeabb664e0e64c8b95)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:57.627426 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:57.629715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:57.676893 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0796890e391a4ffeabb664e0e64c8b95'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.676893 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0796890e391a4ffeabb664e0e64c8b95'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.678795 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9accaf-b1cd-4355-ae2c-a300dc28939f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:57.680270 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9accaf-b1cd-4355-ae2c-a300dc28939f service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:57.689789 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 902/1829] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:57.690152 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0796890e391a4ffeabb664e0e64c8b95'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.690688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9accaf-b1cd-4355-ae2c-a300dc28939f service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:57.697547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0796890e391a4ffeabb664e0e64c8b95'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.698260 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0796890e391a4ffeabb664e0e64c8b95'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:57.700027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9accaf-b1cd-4355-ae2c-a300dc28939f service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:57.706471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5009efd-9ad4-4620-8968-eb5080c38a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:57.707389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5009efd-9ad4-4620-8968-eb5080c38a98 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.707819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5009efd-9ad4-4620-8968-eb5080c38a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.708231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5009efd-9ad4-4620-8968-eb5080c38a98 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.709242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5009efd-9ad4-4620-8968-eb5080c38a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:57.711492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5009efd-9ad4-4620-8968-eb5080c38a98 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:57.717976 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 928/1830] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:57 2022] DELETE /identity/v3/users/0796890e391a4ffeabb664e0e64c8b95 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:57.719216 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:57.720836 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:57.755081 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 903/1831] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:57 2022] POST /identity/v3/users => generated 300 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:57.764085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5009efd-9ad4-4620-8968-eb5080c38a98 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:57.766203 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c5009efd-9ad4-4620-8968-eb5080c38a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:57.794847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5009efd-9ad4-4620-8968-eb5080c38a98 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:57.811778 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:57.866694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b22ea3-22ce-4e1e-9f3a-d1bbcfaedc96 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:57.881022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e388f3a-c4a2-4eb3-ad20-e050d4869d01 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a932c0c8c742eea2200a87320c50a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '247c43ecafa346658e71319e1e2589a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:57.881969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e388f3a-c4a2-4eb3-ad20-e050d4869d01 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.882400 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e388f3a-c4a2-4eb3-ad20-e050d4869d01 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.882876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e388f3a-c4a2-4eb3-ad20-e050d4869d01 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.883772 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e388f3a-c4a2-4eb3-ad20-e050d4869d01 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:57.885807 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e388f3a-c4a2-4eb3-ad20-e050d4869d01 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:57.919249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e55677-725b-4300-9c3c-edfedd4546c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:57.920080 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e55677-725b-4300-9c3c-edfedd4546c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:57.920456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e55677-725b-4300-9c3c-edfedd4546c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:57.920828 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e55677-725b-4300-9c3c-edfedd4546c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:57.921821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e55677-725b-4300-9c3c-edfedd4546c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:57.924312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e55677-725b-4300-9c3c-edfedd4546c4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:57.945242 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 929/1832] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:57 2022] POST /identity/v3/projects => generated 354 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:57.951887 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 904/1833] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:57.952050 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e388f3a-c4a2-4eb3-ad20-e050d4869d01 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:57.956886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e55677-725b-4300-9c3c-edfedd4546c4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:57.980414 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e55677-725b-4300-9c3c-edfedd4546c4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:58.115353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e388f3a-c4a2-4eb3-ad20-e050d4869d01 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:58.141688 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5b7be-e529-4776-ab9d-70dab7a1c4a3 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a932c0c8c742eea2200a87320c50a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '247c43ecafa346658e71319e1e2589a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:58.141944 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205103e0-898b-4afe-a187-89d96baac4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:58.143055 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205103e0-898b-4afe-a187-89d96baac4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:58.143339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205103e0-898b-4afe-a187-89d96baac4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:58.144820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205103e0-898b-4afe-a187-89d96baac4c1 admin admin] PATH_INFO: `/v3/projects/26908c523473402498b0b22cb10715d4/users/54e8d0f872b140f3a25f63c7ea1dcb4e/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:58.145512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5b7be-e529-4776-ab9d-70dab7a1c4a3 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:58.145987 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5b7be-e529-4776-ab9d-70dab7a1c4a3 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:58.146486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5b7be-e529-4776-ab9d-70dab7a1c4a3 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:58.147835 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5b7be-e529-4776-ab9d-70dab7a1c4a3 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:58.148342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5b7be-e529-4776-ab9d-70dab7a1c4a3 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:58.151037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5b7be-e529-4776-ab9d-70dab7a1c4a3 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:58.172051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:58.180739 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 930/1834] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:58.185296 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205103e0-898b-4afe-a187-89d96baac4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26908c523473402498b0b22cb10715d4, user_id=54e8d0f872b140f3a25f63c7ea1dcb4e, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:58.188211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205103e0-898b-4afe-a187-89d96baac4c1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:58.211096 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 905/1835] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:57 2022] PUT /identity/v3/projects/26908c523473402498b0b22cb10715d4/users/54e8d0f872b140f3a25f63c7ea1dcb4e/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:58.215008 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5b7be-e529-4776-ab9d-70dab7a1c4a3 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:58.230537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5b7be-e529-4776-ab9d-70dab7a1c4a3 tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:58.232700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205103e0-898b-4afe-a187-89d96baac4c1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:58.306211 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce3337-bede-493a-b4c4-23192ed53c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:58.307825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce3337-bede-493a-b4c4-23192ed53c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:58.308377 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce3337-bede-493a-b4c4-23192ed53c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:58.308670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce3337-bede-493a-b4c4-23192ed53c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:58.309509 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce3337-bede-493a-b4c4-23192ed53c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:58.311719 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce3337-bede-493a-b4c4-23192ed53c5a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:58.326372 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 931/1836] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:58.327491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205103e0-898b-4afe-a187-89d96baac4c1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:58.327862 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce3337-bede-493a-b4c4-23192ed53c5a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:58.334001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a932c0c8c742eea2200a87320c50a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '247c43ecafa346658e71319e1e2589a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:58.335268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:58.336012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:58.336012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] PATH_INFO: `/v3/projects/68cfc4d1ef41464c93df53237e5d32b6` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:58.337435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: Authorizing `identity:delete_project(project_id=68cfc4d1ef41464c93df53237e5d32b6)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:58.339512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce3337-bede-493a-b4c4-23192ed53c5a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:58.339885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:58.344533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c6fcd-30da-4c05-980b-dd5a037a3cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:58.346016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c6fcd-30da-4c05-980b-dd5a037a3cda admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:58.346557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c6fcd-30da-4c05-980b-dd5a037a3cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:58.346659 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c6fcd-30da-4c05-980b-dd5a037a3cda admin admin] PATH_INFO: `/v3/projects/26908c523473402498b0b22cb10715d4/users/54e8d0f872b140f3a25f63c7ea1dcb4e/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:58.348767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c6fcd-30da-4c05-980b-dd5a037a3cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=26908c523473402498b0b22cb10715d4, user_id=54e8d0f872b140f3a25f63c7ea1dcb4e, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:58.351546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c6fcd-30da-4c05-980b-dd5a037a3cda admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:58.366362 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 932/1837] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:12:58 2022] PUT /identity/v3/projects/26908c523473402498b0b22cb10715d4/users/54e8d0f872b140f3a25f63c7ea1dcb4e/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:58.374479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7827b-598f-4da9-a0b2-5785e0811e4f None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:58.374750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7827b-598f-4da9-a0b2-5785e0811e4f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:58.375201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7827b-598f-4da9-a0b2-5785e0811e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:58.410507 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:58.411135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-1ec7827b-598f-4da9-a0b2-5785e0811e4f None None] MFA Rules not processed for user `54e8d0f872b140f3a25f63c7ea1dcb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:58.412766 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1ec7827b-598f-4da9-a0b2-5785e0811e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:58.426315 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68cfc4d1ef41464c93df53237e5d32b6', 'request_id': 'req-279cefb7-be6e-4711-bb90-8d9a9f46f19f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.437341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68cfc4d1ef41464c93df53237e5d32b6', 'request_id': 'req-279cefb7-be6e-4711-bb90-8d9a9f46f19f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.461771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68cfc4d1ef41464c93df53237e5d32b6', 'request_id': 'req-279cefb7-be6e-4711-bb90-8d9a9f46f19f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.487304 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] The token cache is being invalidate because project 68cfc4d1ef41464c93df53237e5d32b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:58.489610 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.490582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.495433 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 933/1838] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:58.509315 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 906/1839] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:58 2022] DELETE /identity/v3/projects/68cfc4d1ef41464c93df53237e5d32b6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:12:58.510690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7827b-598f-4da9-a0b2-5785e0811e4f None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:58.512735 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1ec7827b-598f-4da9-a0b2-5785e0811e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:58.532580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:58.535495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:58.646366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7827b-598f-4da9-a0b2-5785e0811e4f None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:58.650765 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ece1da-e488-4267-8494-3ec274974d07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:58.651454 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ece1da-e488-4267-8494-3ec274974d07 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:58.651860 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ece1da-e488-4267-8494-3ec274974d07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:58.652159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ece1da-e488-4267-8494-3ec274974d07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:58.655037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-94ece1da-e488-4267-8494-3ec274974d07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:58.668809 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cefb7-be6e-4711-bb90-8d9a9f46f19f tempest-VolumeQuotaClassesTest-758085162 tempest-VolumeQuotaClassesTest-758085162-project-admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:58.669005 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ece1da-e488-4267-8494-3ec274974d07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:58.670559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ece1da-e488-4267-8494-3ec274974d07 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:58.675435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:58.676456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:58.676932 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:58.677308 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] PATH_INFO: `/v3/projects/d802936f8ae940ea86161b4b3bc2e195` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:58.679062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d802936f8ae940ea86161b4b3bc2e195)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:58.681311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:58.681443 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 934/1840] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:12:58.682309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ece1da-e488-4267-8494-3ec274974d07 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:58.683636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-94ece1da-e488-4267-8494-3ec274974d07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:58.697562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ece1da-e488-4267-8494-3ec274974d07 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:58.703293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:58.704301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:58.705109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:58.705400 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] PATH_INFO: `/v3/projects/24e10bfffb814ed2ad0de965b84ea08d` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:58.706780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24e10bfffb814ed2ad0de965b84ea08d)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:58.710577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:58.733816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:58.740850 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd802936f8ae940ea86161b4b3bc2e195', 'request_id': 'req-4b8df11f-bdb4-4c88-a912-2204408991b4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.751072 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd802936f8ae940ea86161b4b3bc2e195', 'request_id': 'req-4b8df11f-bdb4-4c88-a912-2204408991b4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.764651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd802936f8ae940ea86161b4b3bc2e195', 'request_id': 'req-4b8df11f-bdb4-4c88-a912-2204408991b4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.779817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] The token cache is being invalidate because project d802936f8ae940ea86161b4b3bc2e195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:58.779937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.781037 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.783180 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 907/1841] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:58 2022] DELETE /identity/v3/projects/d802936f8ae940ea86161b4b3bc2e195 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:12:58.785398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:58.787139 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:58.824066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e10bfffb814ed2ad0de965b84ea08d', 'request_id': 'req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.838953 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e10bfffb814ed2ad0de965b84ea08d', 'request_id': 'req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.853967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e10bfffb814ed2ad0de965b84ea08d', 'request_id': 'req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.875418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:58.879920 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] The token cache is being invalidate because project 24e10bfffb814ed2ad0de965b84ea08d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:58.880415 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.881207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:58.883461 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 935/1842] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:58 2022] DELETE /identity/v3/projects/24e10bfffb814ed2ad0de965b84ea08d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:58.884320 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:58.886151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:58.900989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8df11f-bdb4-4c88-a912-2204408991b4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:58.971787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4afe4-086a-4df3-853d-fe4ed9e42aa4 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:58.975765 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:58.976697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:58.976813 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050a0dd-3e37-4f2a-8d24-45b298ac319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:58.976961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:58.977401 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] PATH_INFO: `/v3/users/68d5f03462754e4e91328dfb84971c90` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:58.977953 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a0dd-3e37-4f2a-8d24-45b298ac319d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:58.978447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a0dd-3e37-4f2a-8d24-45b298ac319d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:58.978742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] RBAC: Authorizing `identity:delete_user(user_id=68d5f03462754e4e91328dfb84971c90)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:58.978890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a0dd-3e37-4f2a-8d24-45b298ac319d admin admin] PATH_INFO: `/v3/users/5bc768ad1b43466f966b033b779660fb` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:58.980276 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050a0dd-3e37-4f2a-8d24-45b298ac319d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc768ad1b43466f966b033b779660fb)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:58.980798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:58.982341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050a0dd-3e37-4f2a-8d24-45b298ac319d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:59.042796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d050a0dd-3e37-4f2a-8d24-45b298ac319d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc768ad1b43466f966b033b779660fb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.043606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d050a0dd-3e37-4f2a-8d24-45b298ac319d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc768ad1b43466f966b033b779660fb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.043813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68d5f03462754e4e91328dfb84971c90'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.044479 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68d5f03462754e4e91328dfb84971c90'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.045209 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68d5f03462754e4e91328dfb84971c90'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.054287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68d5f03462754e4e91328dfb84971c90'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.065281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d050a0dd-3e37-4f2a-8d24-45b298ac319d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc768ad1b43466f966b033b779660fb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.074322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68d5f03462754e4e91328dfb84971c90'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.076004 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d050a0dd-3e37-4f2a-8d24-45b298ac319d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc768ad1b43466f966b033b779660fb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.076884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d050a0dd-3e37-4f2a-8d24-45b298ac319d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc768ad1b43466f966b033b779660fb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.092577 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 908/1843] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:58 2022] DELETE /identity/v3/users/68d5f03462754e4e91328dfb84971c90 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:59.093679 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:59.095491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:59.117106 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 936/1844] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:58 2022] DELETE /identity/v3/users/5bc768ad1b43466f966b033b779660fb => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:59.126791 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866e717-d0dd-4e2f-bf89-436b423096d1 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.127100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866e717-d0dd-4e2f-bf89-436b423096d1 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.127584 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866e717-d0dd-4e2f-bf89-436b423096d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.157351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3866e717-d0dd-4e2f-bf89-436b423096d1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:59.158976 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3866e717-d0dd-4e2f-bf89-436b423096d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:59.177992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:59.189907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fbac9-e656-47cb-a9e7-38fc616b543a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:59.235491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:59.254620 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 937/1845] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:59.254620 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00dd4-4587-406b-bcc1-d1bac59031ac None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.254620 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00dd4-4587-406b-bcc1-d1bac59031ac None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.254620 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00dd4-4587-406b-bcc1-d1bac59031ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.282587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-bfb00dd4-4587-406b-bcc1-d1bac59031ac None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:59.283817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-bfb00dd4-4587-406b-bcc1-d1bac59031ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:59.295242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:59.296040 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.296399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.296750 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] PATH_INFO: `/v3/users/654a9b1ae9c147a09d890a879f5d95f5` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.299644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=654a9b1ae9c147a09d890a879f5d95f5)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:59.302085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:59.304451 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 938/1846] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:59.306066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7635e-45a7-4106-a2b4-55d9072964e1 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.306453 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7635e-45a7-4106-a2b4-55d9072964e1 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.306668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7635e-45a7-4106-a2b4-55d9072964e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.325835 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3ea7635e-45a7-4106-a2b4-55d9072964e1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:59.327080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3ea7635e-45a7-4106-a2b4-55d9072964e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:59.340096 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 939/1847] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:59.340096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7635e-45a7-4106-a2b4-55d9072964e1 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:59.342158 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3ea7635e-45a7-4106-a2b4-55d9072964e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:59.365697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '654a9b1ae9c147a09d890a879f5d95f5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.366909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '654a9b1ae9c147a09d890a879f5d95f5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.368151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '654a9b1ae9c147a09d890a879f5d95f5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.380309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '654a9b1ae9c147a09d890a879f5d95f5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.403994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '654a9b1ae9c147a09d890a879f5d95f5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.421228 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7635e-45a7-4106-a2b4-55d9072964e1 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:59.425505 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 909/1848] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:12:59 2022] DELETE /identity/v3/users/654a9b1ae9c147a09d890a879f5d95f5 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:59.425671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d564e-0ef0-42c6-9042-480da88c01a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:59.426339 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d564e-0ef0-42c6-9042-480da88c01a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.426573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:59.426692 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d564e-0ef0-42c6-9042-480da88c01a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.426930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d564e-0ef0-42c6-9042-480da88c01a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.429279 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:59.429399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-863d564e-0ef0-42c6-9042-480da88c01a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:59.443457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d564e-0ef0-42c6-9042-480da88c01a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:59.445471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d564e-0ef0-42c6-9042-480da88c01a6 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:59.458033 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 940/1849] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:59.459517 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de7bc-0ede-4ad1-ad83-bb1ffb84b27b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.460110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de7bc-0ede-4ad1-ad83-bb1ffb84b27b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.460393 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de7bc-0ede-4ad1-ad83-bb1ffb84b27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.479899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e89de7bc-0ede-4ad1-ad83-bb1ffb84b27b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:59.481967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e89de7bc-0ede-4ad1-ad83-bb1ffb84b27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:59.529385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae689d78-784f-4e6f-8d22-093c11e4a653 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:59.536492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e40c3-e3d8-460e-aa19-b63a84c66de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:59.537676 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e40c3-e3d8-460e-aa19-b63a84c66de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.538560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e40c3-e3d8-460e-aa19-b63a84c66de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.539182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e40c3-e3d8-460e-aa19-b63a84c66de8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.540315 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e40c3-e3d8-460e-aa19-b63a84c66de8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:12:59.542054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e40c3-e3d8-460e-aa19-b63a84c66de8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:59.545653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e40c3-e3d8-460e-aa19-b63a84c66de8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:59.561731 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 941/1850] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:12:59.571915 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 910/1851] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:12:59.576725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89de7bc-0ede-4ad1-ad83-bb1ffb84b27b None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:59.580246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e40c3-e3d8-460e-aa19-b63a84c66de8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:59.582411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2c8e40c3-e3d8-460e-aa19-b63a84c66de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:59.587124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89de7bc-0ede-4ad1-ad83-bb1ffb84b27b None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:59.592566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a127398-3710-416c-8ee2-6c2d2bfc316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:59.593244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a127398-3710-416c-8ee2-6c2d2bfc316e admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.593526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a127398-3710-416c-8ee2-6c2d2bfc316e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.594099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a127398-3710-416c-8ee2-6c2d2bfc316e admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.594620 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a127398-3710-416c-8ee2-6c2d2bfc316e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:59.596561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a127398-3710-416c-8ee2-6c2d2bfc316e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:59.625608 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 942/1852] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:12:59 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:59.629827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a127398-3710-416c-8ee2-6c2d2bfc316e admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:59.631754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5a127398-3710-416c-8ee2-6c2d2bfc316e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:59.641442 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:59.662266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e40c3-e3d8-460e-aa19-b63a84c66de8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:59.671268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:59.725609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d51947-95e7-45de-a21b-466d99e0d287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:59.726551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d51947-95e7-45de-a21b-466d99e0d287 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.727391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d51947-95e7-45de-a21b-466d99e0d287 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.727391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d51947-95e7-45de-a21b-466d99e0d287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.734801 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a127398-3710-416c-8ee2-6c2d2bfc316e admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:59.736507 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:59.737469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.738370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.738656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] PATH_INFO: `/v3/projects/66a5947cfaa447e99a2aa0187ea7672a` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.739001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d51947-95e7-45de-a21b-466d99e0d287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:59.739875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66a5947cfaa447e99a2aa0187ea7672a)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:59.740508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d51947-95e7-45de-a21b-466d99e0d287 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:59.741859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:59.763251 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 911/1853] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:12:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:12:59.766396 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d51947-95e7-45de-a21b-466d99e0d287 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:59.777242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d51947-95e7-45de-a21b-466d99e0d287 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:59.785776 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681995f-71a3-4c7b-a266-3c39b7bd4f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:59.787426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681995f-71a3-4c7b-a266-3c39b7bd4f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.787850 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681995f-71a3-4c7b-a266-3c39b7bd4f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.788202 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681995f-71a3-4c7b-a266-3c39b7bd4f40 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.790238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681995f-71a3-4c7b-a266-3c39b7bd4f40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:59.791285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681995f-71a3-4c7b-a266-3c39b7bd4f40 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:59.814657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66a5947cfaa447e99a2aa0187ea7672a', 'request_id': 'req-1c84dc70-f24b-4f61-97df-29a7bcb82c60'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.824961 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 912/1854] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:12:59 2022] POST /identity/v3/users => generated 286 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:12:59.826164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66a5947cfaa447e99a2aa0187ea7672a', 'request_id': 'req-1c84dc70-f24b-4f61-97df-29a7bcb82c60'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.826466 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681995f-71a3-4c7b-a266-3c39b7bd4f40 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:59.839724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66a5947cfaa447e99a2aa0187ea7672a', 'request_id': 'req-1c84dc70-f24b-4f61-97df-29a7bcb82c60'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.844096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681995f-71a3-4c7b-a266-3c39b7bd4f40 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:12:59.866680 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] The token cache is being invalidate because project 66a5947cfaa447e99a2aa0187ea7672a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:12:59.867074 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.867927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1c84dc70-f24b-4f61-97df-29a7bcb82c60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:12:59.870030 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 943/1855] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:59 2022] DELETE /identity/v3/projects/66a5947cfaa447e99a2aa0187ea7672a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:12:59.875902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc72f6c-dd52-4058-a947-3914e5772bf4 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.876385 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc72f6c-dd52-4058-a947-3914e5772bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.877665 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc72f6c-dd52-4058-a947-3914e5772bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.906187 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-0dc72f6c-dd52-4058-a947-3914e5772bf4 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:12:59.907563 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0dc72f6c-dd52-4058-a947-3914e5772bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:12:59.942065 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f183e22-c34a-4dbb-b852-3672a39e9025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:12:59.943185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f183e22-c34a-4dbb-b852-3672a39e9025 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:12:59.943476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f183e22-c34a-4dbb-b852-3672a39e9025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:12:59.943810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f183e22-c34a-4dbb-b852-3672a39e9025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:12:59.945039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f183e22-c34a-4dbb-b852-3672a39e9025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:12:59.948597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f183e22-c34a-4dbb-b852-3672a39e9025 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:12:59.975934 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 913/1856] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:12:59.977168 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f183e22-c34a-4dbb-b852-3672a39e9025 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:12:59.980337 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7f183e22-c34a-4dbb-b852-3672a39e9025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.001598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f183e22-c34a-4dbb-b852-3672a39e9025 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.001682 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 944/1857] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:00.001682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc72f6c-dd52-4058-a947-3914e5772bf4 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.001682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0dc72f6c-dd52-4058-a947-3914e5772bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.008757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.009699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.010040 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.010772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] PATH_INFO: `/v3/projects/ff6dcf0f79f3441895c662a922c80896` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.012149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff6dcf0f79f3441895c662a922c80896)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.014138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.016922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc72f6c-dd52-4058-a947-3914e5772bf4 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.022896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eec492-b9c1-4c7d-a55c-baf9ae8a3176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.023898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eec492-b9c1-4c7d-a55c-baf9ae8a3176 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.024468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eec492-b9c1-4c7d-a55c-baf9ae8a3176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.025194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eec492-b9c1-4c7d-a55c-baf9ae8a3176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.026007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eec492-b9c1-4c7d-a55c-baf9ae8a3176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.028207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eec492-b9c1-4c7d-a55c-baf9ae8a3176 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.044691 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 945/1858] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:00.044691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a59995-faa9-4771-b422-523a2f05c523 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.044691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a59995-faa9-4771-b422-523a2f05c523 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.044691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a59995-faa9-4771-b422-523a2f05c523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.063583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-11a59995-faa9-4771-b422-523a2f05c523 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:00.065286 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-11a59995-faa9-4771-b422-523a2f05c523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.074561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff6dcf0f79f3441895c662a922c80896', 'request_id': 'req-3416e741-28f7-4c06-9204-d8544bfd818a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.084678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff6dcf0f79f3441895c662a922c80896', 'request_id': 'req-3416e741-28f7-4c06-9204-d8544bfd818a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.093445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff6dcf0f79f3441895c662a922c80896', 'request_id': 'req-3416e741-28f7-4c06-9204-d8544bfd818a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.105385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:00.109701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] The token cache is being invalidate because project ff6dcf0f79f3441895c662a922c80896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:00.109998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.111114 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.113157 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 914/1859] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:12:59 2022] DELETE /identity/v3/projects/ff6dcf0f79f3441895c662a922c80896 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:00.113497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:00.117044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.120828 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.141861 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 946/1860] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:00.142852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a59995-faa9-4771-b422-523a2f05c523 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.144153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-11a59995-faa9-4771-b422-523a2f05c523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.156427 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a59995-faa9-4771-b422-523a2f05c523 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.161327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.162249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.162544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.162826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] PATH_INFO: `/v3/projects/021fea6560f949b082660d458130c7b5` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.164564 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=021fea6560f949b082660d458130c7b5)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.165930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.204448 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416e741-28f7-4c06-9204-d8544bfd818a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.209475 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcc463-220e-4274-af1a-6e2a188a0443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.210796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcc463-220e-4274-af1a-6e2a188a0443 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.211021 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcc463-220e-4274-af1a-6e2a188a0443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.211305 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcc463-220e-4274-af1a-6e2a188a0443 admin admin] PATH_INFO: `/v3/projects/edea0a11fe3843b2bde7d7b97d92c661/users/8e35a25becf842ae9cd9323e8152edb9/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.224694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '021fea6560f949b082660d458130c7b5', 'request_id': 'req-2f191213-7abc-43be-a8fb-4177cb4ae330'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.230632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcc463-220e-4274-af1a-6e2a188a0443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edea0a11fe3843b2bde7d7b97d92c661, user_id=8e35a25becf842ae9cd9323e8152edb9, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.231368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '021fea6560f949b082660d458130c7b5', 'request_id': 'req-2f191213-7abc-43be-a8fb-4177cb4ae330'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.233455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcc463-220e-4274-af1a-6e2a188a0443 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.237822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '021fea6560f949b082660d458130c7b5', 'request_id': 'req-2f191213-7abc-43be-a8fb-4177cb4ae330'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.254642 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] The token cache is being invalidate because project 021fea6560f949b082660d458130c7b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:00.254943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.255589 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.257349 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 947/1861] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:00 2022] DELETE /identity/v3/projects/021fea6560f949b082660d458130c7b5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:00.257938 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.264556 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 915/1862] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:00 2022] PUT /identity/v3/projects/edea0a11fe3843b2bde7d7b97d92c661/users/8e35a25becf842ae9cd9323e8152edb9/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:00.264556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcc463-220e-4274-af1a-6e2a188a0443 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.265371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.265589 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-31dcc463-220e-4274-af1a-6e2a188a0443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.337457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcc463-220e-4274-af1a-6e2a188a0443 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.338455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f191213-7abc-43be-a8fb-4177cb4ae330 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.343159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f70823-da4e-4aa5-af2d-535184d856aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.344043 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f70823-da4e-4aa5-af2d-535184d856aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.344318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f70823-da4e-4aa5-af2d-535184d856aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.344587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f70823-da4e-4aa5-af2d-535184d856aa admin admin] PATH_INFO: `/v3/users/54e8d0f872b140f3a25f63c7ea1dcb4e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.345837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f70823-da4e-4aa5-af2d-535184d856aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e8d0f872b140f3a25f63c7ea1dcb4e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.347799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f70823-da4e-4aa5-af2d-535184d856aa admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.349130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24ac78-f515-4a50-89d2-f58e880d50cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.350488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24ac78-f515-4a50-89d2-f58e880d50cb admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.350488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24ac78-f515-4a50-89d2-f58e880d50cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.350774 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24ac78-f515-4a50-89d2-f58e880d50cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.351608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24ac78-f515-4a50-89d2-f58e880d50cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:00.351938 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24ac78-f515-4a50-89d2-f58e880d50cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.353546 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24ac78-f515-4a50-89d2-f58e880d50cb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.366532 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 916/1863] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:00.367394 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24ac78-f515-4a50-89d2-f58e880d50cb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.369270 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fd24ac78-f515-4a50-89d2-f58e880d50cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.383068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24ac78-f515-4a50-89d2-f58e880d50cb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.387834 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.388672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.388957 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.389234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] PATH_INFO: `/v3/users/07a932c0c8c742eea2200a87320c50a4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.390464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] RBAC: Authorizing `identity:delete_user(user_id=07a932c0c8c742eea2200a87320c50a4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.392465 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.403161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-41f70823-da4e-4aa5-af2d-535184d856aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e8d0f872b140f3a25f63c7ea1dcb4e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.404726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-41f70823-da4e-4aa5-af2d-535184d856aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e8d0f872b140f3a25f63c7ea1dcb4e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.420426 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-41f70823-da4e-4aa5-af2d-535184d856aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e8d0f872b140f3a25f63c7ea1dcb4e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.422814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:00.431280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-41f70823-da4e-4aa5-af2d-535184d856aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e8d0f872b140f3a25f63c7ea1dcb4e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.432251 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-41f70823-da4e-4aa5-af2d-535184d856aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e8d0f872b140f3a25f63c7ea1dcb4e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.447479 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 948/1864] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:00 2022] DELETE /identity/v3/users/54e8d0f872b140f3a25f63c7ea1dcb4e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:00.450988 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07a932c0c8c742eea2200a87320c50a4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.451833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07a932c0c8c742eea2200a87320c50a4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.452543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07a932c0c8c742eea2200a87320c50a4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.456034 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adb117-f771-4cda-a3fa-7dee5d4ee3c5 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.456600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adb117-f771-4cda-a3fa-7dee5d4ee3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.457024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adb117-f771-4cda-a3fa-7dee5d4ee3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.461249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07a932c0c8c742eea2200a87320c50a4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.481381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:00.483631 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07a932c0c8c742eea2200a87320c50a4'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:00.488451 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-98adb117-f771-4cda-a3fa-7dee5d4ee3c5 None None] MFA Rules not processed for user `8e35a25becf842ae9cd9323e8152edb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:00.490214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-98adb117-f771-4cda-a3fa-7dee5d4ee3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.507710 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 917/1865] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:00 2022] DELETE /identity/v3/users/07a932c0c8c742eea2200a87320c50a4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:00.513627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.516042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.601530 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 949/1866] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:00.611832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476b0c4-c968-4d0b-b31e-98e0d668732e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.615760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adb117-f771-4cda-a3fa-7dee5d4ee3c5 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.617441 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-98adb117-f771-4cda-a3fa-7dee5d4ee3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.619208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4504f16-e837-4231-a436-a7d719c79f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.620244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4504f16-e837-4231-a436-a7d719c79f5e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.620568 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4504f16-e837-4231-a436-a7d719c79f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.620978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4504f16-e837-4231-a436-a7d719c79f5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.622598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4504f16-e837-4231-a436-a7d719c79f5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.625277 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4504f16-e837-4231-a436-a7d719c79f5e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.679022 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 918/1867] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:00 2022] POST /identity/v3/projects => generated 409 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:00.683446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4504f16-e837-4231-a436-a7d719c79f5e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.705716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adb117-f771-4cda-a3fa-7dee5d4ee3c5 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.784005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4504f16-e837-4231-a436-a7d719c79f5e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.788109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee68cc6-1072-4e5d-96ff-d184e75015c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.789240 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee68cc6-1072-4e5d-96ff-d184e75015c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.789739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee68cc6-1072-4e5d-96ff-d184e75015c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.790208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee68cc6-1072-4e5d-96ff-d184e75015c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.794646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e5f83-9bd3-4512-bb6c-bcedd692d981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.795555 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e5f83-9bd3-4512-bb6c-bcedd692d981 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.795844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e5f83-9bd3-4512-bb6c-bcedd692d981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.796121 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e5f83-9bd3-4512-bb6c-bcedd692d981 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.796904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e5f83-9bd3-4512-bb6c-bcedd692d981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.804522 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e5f83-9bd3-4512-bb6c-bcedd692d981 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.853848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:00.870107 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 919/1868] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:00 2022] POST /identity/v3/users => generated 305 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:00.875007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e5f83-9bd3-4512-bb6c-bcedd692d981 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.891573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e5f83-9bd3-4512-bb6c-bcedd692d981 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.898171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee68cc6-1072-4e5d-96ff-d184e75015c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.899950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22c3bf-c739-42fd-b769-e9b0e55aace3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.900876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22c3bf-c739-42fd-b769-e9b0e55aace3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.901283 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22c3bf-c739-42fd-b769-e9b0e55aace3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.901819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22c3bf-c739-42fd-b769-e9b0e55aace3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.902436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee68cc6-1072-4e5d-96ff-d184e75015c7 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.903691 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22c3bf-c739-42fd-b769-e9b0e55aace3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.907512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22c3bf-c739-42fd-b769-e9b0e55aace3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.922113 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 920/1869] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:00.927341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22c3bf-c739-42fd-b769-e9b0e55aace3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.938030 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 950/1870] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:00.942421 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22c3bf-c739-42fd-b769-e9b0e55aace3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.951695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0913b-b38e-40bb-881c-7603b327b2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.952868 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0913b-b38e-40bb-881c-7603b327b2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.953292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0913b-b38e-40bb-881c-7603b327b2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.953668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0913b-b38e-40bb-881c-7603b327b2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:00.954536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0913b-b38e-40bb-881c-7603b327b2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:00.956605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0913b-b38e-40bb-881c-7603b327b2b2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:00.962862 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee68cc6-1072-4e5d-96ff-d184e75015c7 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.964707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-eee68cc6-1072-4e5d-96ff-d184e75015c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:00.977185 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 921/1871] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:00.982595 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0913b-b38e-40bb-881c-7603b327b2b2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:00.988327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee68cc6-1072-4e5d-96ff-d184e75015c7 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:00.994131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09a402-3342-42d7-b999-47c7f714436b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:00.995616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a402-3342-42d7-b999-47c7f714436b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:00.996437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a402-3342-42d7-b999-47c7f714436b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:00.997416 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a402-3342-42d7-b999-47c7f714436b admin admin] PATH_INFO: `/v3/users/8e35a25becf842ae9cd9323e8152edb9` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.001649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09a402-3342-42d7-b999-47c7f714436b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e35a25becf842ae9cd9323e8152edb9)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:01.004284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09a402-3342-42d7-b999-47c7f714436b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:01.036221 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0913b-b38e-40bb-881c-7603b327b2b2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:01.048244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a11f0-8886-49cf-9389-c4a85487c261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:01.050583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a11f0-8886-49cf-9389-c4a85487c261 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.050966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a11f0-8886-49cf-9389-c4a85487c261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.052747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a11f0-8886-49cf-9389-c4a85487c261 admin admin] PATH_INFO: `/v3/projects/7a9fa441b8864879bef5e4d29f1b7fef/users/f14703de0a894961911ccb34ad1036b4/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.096712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a11f0-8886-49cf-9389-c4a85487c261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9fa441b8864879bef5e4d29f1b7fef, user_id=f14703de0a894961911ccb34ad1036b4, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:01.103323 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a11f0-8886-49cf-9389-c4a85487c261 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:01.109982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-3c09a402-3342-42d7-b999-47c7f714436b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e35a25becf842ae9cd9323e8152edb9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.110781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-3c09a402-3342-42d7-b999-47c7f714436b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e35a25becf842ae9cd9323e8152edb9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.136795 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 922/1872] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:00 2022] PUT /identity/v3/projects/7a9fa441b8864879bef5e4d29f1b7fef/users/f14703de0a894961911ccb34ad1036b4/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:01.138167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-3c09a402-3342-42d7-b999-47c7f714436b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e35a25becf842ae9cd9323e8152edb9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.147171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-3c09a402-3342-42d7-b999-47c7f714436b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e35a25becf842ae9cd9323e8152edb9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.148101 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-3c09a402-3342-42d7-b999-47c7f714436b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e35a25becf842ae9cd9323e8152edb9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.149191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea717c-1206-413b-9a36-97f5a700dcef None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.149749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea717c-1206-413b-9a36-97f5a700dcef None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.150657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea717c-1206-413b-9a36-97f5a700dcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.177269 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 951/1873] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:00 2022] DELETE /identity/v3/users/8e35a25becf842ae9cd9323e8152edb9 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:01.183256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79a738-acf1-4885-bafb-dd703a2d0529 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.183985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79a738-acf1-4885-bafb-dd703a2d0529 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.184620 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79a738-acf1-4885-bafb-dd703a2d0529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.206261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-bf79a738-acf1-4885-bafb-dd703a2d0529 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:01.207640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-bf79a738-acf1-4885-bafb-dd703a2d0529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.221772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e5ea717c-1206-413b-9a36-97f5a700dcef None None] MFA Rules not processed for user `f14703de0a894961911ccb34ad1036b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:01.222539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e5ea717c-1206-413b-9a36-97f5a700dcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.231299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:01.334590 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 923/1874] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:01.336036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea717c-1206-413b-9a36-97f5a700dcef None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:01.337770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e5ea717c-1206-413b-9a36-97f5a700dcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.360823 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 952/1875] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:01.360823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79a738-acf1-4885-bafb-dd703a2d0529 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:01.366969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea717c-1206-413b-9a36-97f5a700dcef None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:01.368864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-bf79a738-acf1-4885-bafb-dd703a2d0529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.384313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:01.384794 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79a738-acf1-4885-bafb-dd703a2d0529 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:01.387068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.387652 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.388147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] PATH_INFO: `/v3/projects/247c43ecafa346658e71319e1e2589a5` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.389684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=247c43ecafa346658e71319e1e2589a5)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:01.389889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef86629-90e2-417b-b12e-298a67843d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:01.390621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef86629-90e2-417b-b12e-298a67843d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.390913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef86629-90e2-417b-b12e-298a67843d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.391195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef86629-90e2-417b-b12e-298a67843d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.392384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef86629-90e2-417b-b12e-298a67843d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:01.393571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:01.393890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef86629-90e2-417b-b12e-298a67843d58 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:01.448205 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 953/1876] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:01 2022] POST /identity/v3/projects => generated 409 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:01.449137 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef86629-90e2-417b-b12e-298a67843d58 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:01.453821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0ef86629-90e2-417b-b12e-298a67843d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.473688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '247c43ecafa346658e71319e1e2589a5', 'request_id': 'req-588df89a-3723-4f64-bb5f-2d7c3fba19f2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.495780 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '247c43ecafa346658e71319e1e2589a5', 'request_id': 'req-588df89a-3723-4f64-bb5f-2d7c3fba19f2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.516580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '247c43ecafa346658e71319e1e2589a5', 'request_id': 'req-588df89a-3723-4f64-bb5f-2d7c3fba19f2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.530809 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] The token cache is being invalidate because project 247c43ecafa346658e71319e1e2589a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:01.531424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.532118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:01.533161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.538250 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 924/1877] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:01 2022] DELETE /identity/v3/projects/247c43ecafa346658e71319e1e2589a5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:01.538250 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:01.539969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.573335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef86629-90e2-417b-b12e-298a67843d58 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:01.640245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588df89a-3723-4f64-bb5f-2d7c3fba19f2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:01.641341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b025e8-c2ca-4491-aea9-241de348c1f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:01.642013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b025e8-c2ca-4491-aea9-241de348c1f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.642410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b025e8-c2ca-4491-aea9-241de348c1f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.642717 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b025e8-c2ca-4491-aea9-241de348c1f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.644580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-51b025e8-c2ca-4491-aea9-241de348c1f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.645345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56479a9e-fb95-4c28-b28c-3b039d984cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:01.646025 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56479a9e-fb95-4c28-b28c-3b039d984cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.646332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56479a9e-fb95-4c28-b28c-3b039d984cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.646605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56479a9e-fb95-4c28-b28c-3b039d984cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.647574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56479a9e-fb95-4c28-b28c-3b039d984cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:01.649521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56479a9e-fb95-4c28-b28c-3b039d984cb7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:01.658371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b025e8-c2ca-4491-aea9-241de348c1f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:01.659918 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b025e8-c2ca-4491-aea9-241de348c1f5 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:01.670614 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 954/1878] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:01.671416 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b025e8-c2ca-4491-aea9-241de348c1f5 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:01.672884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-51b025e8-c2ca-4491-aea9-241de348c1f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.685150 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b025e8-c2ca-4491-aea9-241de348c1f5 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:01.693293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:01.694470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.694618 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 925/1879] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:01 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:01.694710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.694789 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] PATH_INFO: `/v3/projects/26908c523473402498b0b22cb10715d4` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.695984 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26908c523473402498b0b22cb10715d4)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:01.698244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:01.698703 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6063c58-763a-461f-bdea-989431c4a4cd None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.699101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6063c58-763a-461f-bdea-989431c4a4cd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.699386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6063c58-763a-461f-bdea-989431c4a4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.725757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f6063c58-763a-461f-bdea-989431c4a4cd None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:01.727117 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f6063c58-763a-461f-bdea-989431c4a4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.736514 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 926/1880] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:01.737322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6063c58-763a-461f-bdea-989431c4a4cd None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:01.747695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6063c58-763a-461f-bdea-989431c4a4cd None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:01.776878 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:01.782391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26908c523473402498b0b22cb10715d4', 'request_id': 'req-57afcc39-0932-40c5-8159-34d049435776'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.789810 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26908c523473402498b0b22cb10715d4', 'request_id': 'req-57afcc39-0932-40c5-8159-34d049435776'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.799856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26908c523473402498b0b22cb10715d4', 'request_id': 'req-57afcc39-0932-40c5-8159-34d049435776'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.825498 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] The token cache is being invalidate because project 26908c523473402498b0b22cb10715d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:01.825815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.828800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-57afcc39-0932-40c5-8159-34d049435776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:01.834416 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 955/1881] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:01 2022] DELETE /identity/v3/projects/26908c523473402498b0b22cb10715d4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:01.840258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a9bdd-002a-41fc-8431-4a683a3110ab None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.840626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a9bdd-002a-41fc-8431-4a683a3110ab None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.840896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a9bdd-002a-41fc-8431-4a683a3110ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.868982 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a3c70-f7bc-4e00-9c9c-a45514fe10fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:01.870166 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a3c70-f7bc-4e00-9c9c-a45514fe10fe admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.870653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a3c70-f7bc-4e00-9c9c-a45514fe10fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.871297 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a3c70-f7bc-4e00-9c9c-a45514fe10fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.874036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a3c70-f7bc-4e00-9c9c-a45514fe10fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:01.874881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e33a9bdd-002a-41fc-8431-4a683a3110ab None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:01.879841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a3c70-f7bc-4e00-9c9c-a45514fe10fe admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:01.880094 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e33a9bdd-002a-41fc-8431-4a683a3110ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.893507 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 956/1882] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:01.899077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3335f6-f471-4123-b18b-43797f05724f None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.899077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3335f6-f471-4123-b18b-43797f05724f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.899077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3335f6-f471-4123-b18b-43797f05724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.901987 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 927/1883] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:01.903877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a3c70-f7bc-4e00-9c9c-a45514fe10fe admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:01.931762 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a3c70-f7bc-4e00-9c9c-a45514fe10fe admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:01.937151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-9d3335f6-f471-4123-b18b-43797f05724f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:01.938528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a4b4d-0706-43fc-8f0b-f979010381c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:01.939294 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a4b4d-0706-43fc-8f0b-f979010381c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.939688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a4b4d-0706-43fc-8f0b-f979010381c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.939977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a4b4d-0706-43fc-8f0b-f979010381c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.940730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a4b4d-0706-43fc-8f0b-f979010381c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:01.941041 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a4b4d-0706-43fc-8f0b-f979010381c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:01.941744 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9d3335f6-f471-4123-b18b-43797f05724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.942908 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a4b4d-0706-43fc-8f0b-f979010381c8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:01.957357 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 928/1884] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:01.958678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a4b4d-0706-43fc-8f0b-f979010381c8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:01.961728 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2c0a4b4d-0706-43fc-8f0b-f979010381c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:01.961945 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 957/1885] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:01.962979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3335f6-f471-4123-b18b-43797f05724f None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:01.973961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3335f6-f471-4123-b18b-43797f05724f None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:01.978673 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a4b4d-0706-43fc-8f0b-f979010381c8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:01.979674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc5fbb-ad09-4c25-928a-31a7664450ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:01.980476 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc5fbb-ad09-4c25-928a-31a7664450ee admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.980835 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc5fbb-ad09-4c25-928a-31a7664450ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.981211 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc5fbb-ad09-4c25-928a-31a7664450ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:01.982070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc5fbb-ad09-4c25-928a-31a7664450ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:01.984926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc5fbb-ad09-4c25-928a-31a7664450ee admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:01.986938 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c15151-cf2a-4edb-a838-029653107935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:01.988701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c15151-cf2a-4edb-a838-029653107935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:01.989163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c15151-cf2a-4edb-a838-029653107935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:01.989622 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c15151-cf2a-4edb-a838-029653107935 admin admin] PATH_INFO: `/v3/projects/b2531e238d6b4c26800c638330b83acf/users/c937dc53cc7a411badb3e494efa52680/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.017606 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c15151-cf2a-4edb-a838-029653107935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2531e238d6b4c26800c638330b83acf, user_id=c937dc53cc7a411badb3e494efa52680, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.021851 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 958/1886] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:01 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:02.021851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8f14a-722f-46bf-b5ec-c02775749079 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.021851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8f14a-722f-46bf-b5ec-c02775749079 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.022281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c15151-cf2a-4edb-a838-029653107935 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.022381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8f14a-722f-46bf-b5ec-c02775749079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.051614 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 929/1887] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:01 2022] PUT /identity/v3/projects/b2531e238d6b4c26800c638330b83acf/users/c937dc53cc7a411badb3e494efa52680/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:02.052477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c15151-cf2a-4edb-a838-029653107935 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.052965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-52c8f14a-722f-46bf-b5ec-c02775749079 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:02.054256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-52c8f14a-722f-46bf-b5ec-c02775749079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:02.122351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:02.143408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c15151-cf2a-4edb-a838-029653107935 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.148984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01b002-e70b-452b-bcbf-1c51674ce2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.149990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01b002-e70b-452b-bcbf-1c51674ce2fc admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.150312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01b002-e70b-452b-bcbf-1c51674ce2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.150592 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01b002-e70b-452b-bcbf-1c51674ce2fc admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.151271 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 959/1888] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:02.151635 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01b002-e70b-452b-bcbf-1c51674ce2fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.152194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8f14a-722f-46bf-b5ec-c02775749079 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.154701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01b002-e70b-452b-bcbf-1c51674ce2fc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.161744 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8f14a-722f-46bf-b5ec-c02775749079 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.167740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e6b04-3ddb-4731-a50d-23457c88ec36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.168582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e6b04-3ddb-4731-a50d-23457c88ec36 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.168924 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e6b04-3ddb-4731-a50d-23457c88ec36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.169235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e6b04-3ddb-4731-a50d-23457c88ec36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.169977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e6b04-3ddb-4731-a50d-23457c88ec36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.171627 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e6b04-3ddb-4731-a50d-23457c88ec36 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.175919 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 930/1889] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:02 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:02.176767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01b002-e70b-452b-bcbf-1c51674ce2fc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.178366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9f01b002-e70b-452b-bcbf-1c51674ce2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:02.186476 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 960/1890] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:02.186476 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e6b04-3ddb-4731-a50d-23457c88ec36 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.191756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01b002-e70b-452b-bcbf-1c51674ce2fc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.195599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e6b04-3ddb-4731-a50d-23457c88ec36 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.196405 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.197272 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.197556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.197853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] PATH_INFO: `/v3/projects/edea0a11fe3843b2bde7d7b97d92c661` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.199125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edea0a11fe3843b2bde7d7b97d92c661)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.200560 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c990978-17ce-4d7d-8f01-f1dfb4adec58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.201031 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.201345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c990978-17ce-4d7d-8f01-f1dfb4adec58 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.201636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c990978-17ce-4d7d-8f01-f1dfb4adec58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.201917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c990978-17ce-4d7d-8f01-f1dfb4adec58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.202678 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c990978-17ce-4d7d-8f01-f1dfb4adec58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:02.202988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c990978-17ce-4d7d-8f01-f1dfb4adec58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.204570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c990978-17ce-4d7d-8f01-f1dfb4adec58 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.217336 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 961/1891] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:02.219450 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c990978-17ce-4d7d-8f01-f1dfb4adec58 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.230032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c990978-17ce-4d7d-8f01-f1dfb4adec58 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.234978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04c4e1-504a-4ed9-be29-4b2bc75c6af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.235636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04c4e1-504a-4ed9-be29-4b2bc75c6af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.235922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04c4e1-504a-4ed9-be29-4b2bc75c6af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.236205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04c4e1-504a-4ed9-be29-4b2bc75c6af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.236951 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04c4e1-504a-4ed9-be29-4b2bc75c6af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.238659 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04c4e1-504a-4ed9-be29-4b2bc75c6af3 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.248807 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 962/1892] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:02.249178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edea0a11fe3843b2bde7d7b97d92c661', 'request_id': 'req-708b75f9-5dad-41da-9f2a-198cb5f01036'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:02.249657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04c4e1-504a-4ed9-be29-4b2bc75c6af3 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.259674 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edea0a11fe3843b2bde7d7b97d92c661', 'request_id': 'req-708b75f9-5dad-41da-9f2a-198cb5f01036'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:02.259848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04c4e1-504a-4ed9-be29-4b2bc75c6af3 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.261522 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:02.267603 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edea0a11fe3843b2bde7d7b97d92c661', 'request_id': 'req-708b75f9-5dad-41da-9f2a-198cb5f01036'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:02.282012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] The token cache is being invalidate because project edea0a11fe3843b2bde7d7b97d92c661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:02.282277 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:02.282942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:02.284531 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 931/1893] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:02 2022] DELETE /identity/v3/projects/edea0a11fe3843b2bde7d7b97d92c661 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:02.288442 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.288442 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:02.327767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571c0db-6331-419e-bb31-3058db58404e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.328794 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571c0db-6331-419e-bb31-3058db58404e admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.329162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571c0db-6331-419e-bb31-3058db58404e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.329485 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571c0db-6331-419e-bb31-3058db58404e admin admin] PATH_INFO: `/v3/projects/b2531e238d6b4c26800c638330b83acf/users/c937dc53cc7a411badb3e494efa52680/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.331907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571c0db-6331-419e-bb31-3058db58404e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2531e238d6b4c26800c638330b83acf, user_id=c937dc53cc7a411badb3e494efa52680, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.335243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571c0db-6331-419e-bb31-3058db58404e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.346651 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 963/1894] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:02 2022] PUT /identity/v3/projects/b2531e238d6b4c26800c638330b83acf/users/c937dc53cc7a411badb3e494efa52680/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:02.347563 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571c0db-6331-419e-bb31-3058db58404e admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.349223 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5571c0db-6331-419e-bb31-3058db58404e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:02.367470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b75f9-5dad-41da-9f2a-198cb5f01036 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.434778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571c0db-6331-419e-bb31-3058db58404e admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.435484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f2bd8-4032-472c-988e-2178cee0d8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.436239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f2bd8-4032-472c-988e-2178cee0d8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.436671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f2bd8-4032-472c-988e-2178cee0d8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.437050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f2bd8-4032-472c-988e-2178cee0d8f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.438063 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f2bd8-4032-472c-988e-2178cee0d8f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.440172 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f2bd8-4032-472c-988e-2178cee0d8f6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.442586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf3521-0016-4628-9b9a-0e1e7de3b7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.444241 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf3521-0016-4628-9b9a-0e1e7de3b7bb admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.445278 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf3521-0016-4628-9b9a-0e1e7de3b7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.445640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf3521-0016-4628-9b9a-0e1e7de3b7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.446940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf3521-0016-4628-9b9a-0e1e7de3b7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.448726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf3521-0016-4628-9b9a-0e1e7de3b7bb admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.467553 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 964/1895] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:02.467553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b85aa-946b-487e-ab70-1aeed7afd8da None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.467553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b85aa-946b-487e-ab70-1aeed7afd8da None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.467553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b85aa-946b-487e-ab70-1aeed7afd8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.484832 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 932/1896] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:02 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:02.484832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28271c3-9951-4d13-b0ec-9351622f077a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.484832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28271c3-9951-4d13-b0ec-9351622f077a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.484832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28271c3-9951-4d13-b0ec-9351622f077a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.491811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3a0b85aa-946b-487e-ab70-1aeed7afd8da None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:02.493043 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3a0b85aa-946b-487e-ab70-1aeed7afd8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:02.508267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e28271c3-9951-4d13-b0ec-9351622f077a None None] MFA Rules not processed for user `c937dc53cc7a411badb3e494efa52680`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:02.509656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e28271c3-9951-4d13-b0ec-9351622f077a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:02.537840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:02.570960 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 965/1897] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:02.577381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b85aa-946b-487e-ab70-1aeed7afd8da None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.582142 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b85aa-946b-487e-ab70-1aeed7afd8da None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.588383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd9aab-9099-4659-87a9-670dc8616043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.589816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd9aab-9099-4659-87a9-670dc8616043 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.590250 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd9aab-9099-4659-87a9-670dc8616043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.590552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd9aab-9099-4659-87a9-670dc8616043 admin admin] PATH_INFO: `/v3/projects/4a7e4c4e4d154bd9a7197190a9d8b27f/users/c323490c11b9461898687e729cd357f2/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.608923 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd9aab-9099-4659-87a9-670dc8616043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a7e4c4e4d154bd9a7197190a9d8b27f, user_id=c323490c11b9461898687e729cd357f2, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.613190 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 933/1898] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:02 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:02.613190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28271c3-9951-4d13-b0ec-9351622f077a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.613993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd9aab-9099-4659-87a9-670dc8616043 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.623260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:02.629850 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28271c3-9951-4d13-b0ec-9351622f077a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.630029 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 966/1899] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:02 2022] PUT /identity/v3/projects/4a7e4c4e4d154bd9a7197190a9d8b27f/users/c323490c11b9461898687e729cd357f2/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:02.631729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f4184-e28d-46ec-8223-0f242b15257a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.632134 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f4184-e28d-46ec-8223-0f242b15257a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.632436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f4184-e28d-46ec-8223-0f242b15257a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.655272 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-058f4184-e28d-46ec-8223-0f242b15257a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:02.656782 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-058f4184-e28d-46ec-8223-0f242b15257a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:02.707604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2ce71-24f9-466f-a78f-19b2c5e7ed57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.708301 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2ce71-24f9-466f-a78f-19b2c5e7ed57 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.708586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2ce71-24f9-466f-a78f-19b2c5e7ed57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.708865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2ce71-24f9-466f-a78f-19b2c5e7ed57 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.709564 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2ce71-24f9-466f-a78f-19b2c5e7ed57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.712093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2ce71-24f9-466f-a78f-19b2c5e7ed57 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.721468 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 967/1900] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:02.725586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f4184-e28d-46ec-8223-0f242b15257a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.727068 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-058f4184-e28d-46ec-8223-0f242b15257a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:02.733282 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 934/1901] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:02 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:02.735056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762b938-dd1a-4f06-9255-2ecfd0113d4b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.735364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762b938-dd1a-4f06-9255-2ecfd0113d4b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.735644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762b938-dd1a-4f06-9255-2ecfd0113d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.753712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b762b938-dd1a-4f06-9255-2ecfd0113d4b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:02.754971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b762b938-dd1a-4f06-9255-2ecfd0113d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:02.763813 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 935/1902] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:02.764427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762b938-dd1a-4f06-9255-2ecfd0113d4b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.774615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762b938-dd1a-4f06-9255-2ecfd0113d4b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.779808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418945a8-b21d-4728-ac46-03ce39b24a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.780799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418945a8-b21d-4728-ac46-03ce39b24a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.781084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418945a8-b21d-4728-ac46-03ce39b24a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.781420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418945a8-b21d-4728-ac46-03ce39b24a6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.782360 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418945a8-b21d-4728-ac46-03ce39b24a6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:02.782672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418945a8-b21d-4728-ac46-03ce39b24a6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.784327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418945a8-b21d-4728-ac46-03ce39b24a6c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.794287 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f4184-e28d-46ec-8223-0f242b15257a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.796572 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 936/1903] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:02.800012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418945a8-b21d-4728-ac46-03ce39b24a6c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.801452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee9486-fecb-4fe8-9f8e-c77361df8ca4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.802150 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9486-fecb-4fe8-9f8e-c77361df8ca4 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.802533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9486-fecb-4fe8-9f8e-c77361df8ca4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.802739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9486-fecb-4fe8-9f8e-c77361df8ca4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.807428 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418945a8-b21d-4728-ac46-03ce39b24a6c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.812462 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df58d8e-181e-489a-87ff-34062053ba7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.813122 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df58d8e-181e-489a-87ff-34062053ba7c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.813406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df58d8e-181e-489a-87ff-34062053ba7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.813710 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df58d8e-181e-489a-87ff-34062053ba7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.814474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df58d8e-181e-489a-87ff-34062053ba7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.816067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df58d8e-181e-489a-87ff-34062053ba7c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.827098 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 937/1904] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:02.828185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df58d8e-181e-489a-87ff-34062053ba7c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.829958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9df58d8e-181e-489a-87ff-34062053ba7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:02.864843 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee9486-fecb-4fe8-9f8e-c77361df8ca4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.866348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee9486-fecb-4fe8-9f8e-c77361df8ca4 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.875992 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 968/1905] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:02 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:02.876796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee9486-fecb-4fe8-9f8e-c77361df8ca4 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.883454 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:02.888084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee9486-fecb-4fe8-9f8e-c77361df8ca4 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.889839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df58d8e-181e-489a-87ff-34062053ba7c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:02.894438 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a3d53-b028-4c49-b7ff-2d593a843912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.895084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911460e2-34ef-4171-acba-39547893e93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:02.895250 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a3d53-b028-4c49-b7ff-2d593a843912 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.895384 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a3d53-b028-4c49-b7ff-2d593a843912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.895667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a3d53-b028-4c49-b7ff-2d593a843912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.896425 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911460e2-34ef-4171-acba-39547893e93f admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:02.896855 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911460e2-34ef-4171-acba-39547893e93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:02.897392 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911460e2-34ef-4171-acba-39547893e93f admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:02.898408 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911460e2-34ef-4171-acba-39547893e93f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.900531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911460e2-34ef-4171-acba-39547893e93f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.908277 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a3d53-b028-4c49-b7ff-2d593a843912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:02.909785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a3d53-b028-4c49-b7ff-2d593a843912 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:02.938621 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 938/1906] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:02.942577 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 969/1907] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:02 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:02.943127 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a3d53-b028-4c49-b7ff-2d593a843912 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.947610 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911460e2-34ef-4171-acba-39547893e93f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:02.960311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a3d53-b028-4c49-b7ff-2d593a843912 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:03.034810 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911460e2-34ef-4171-acba-39547893e93f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:03.040839 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6b77d-d86e-454d-b4b6-c2781470ccb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:03.042769 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6b77d-d86e-454d-b4b6-c2781470ccb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:03.043081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6b77d-d86e-454d-b4b6-c2781470ccb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:03.043680 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6b77d-d86e-454d-b4b6-c2781470ccb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:03.044586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6b77d-d86e-454d-b4b6-c2781470ccb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:03.046806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6b77d-d86e-454d-b4b6-c2781470ccb1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:03.097478 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 970/1908] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:02 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:03.100193 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59596d-3998-47f6-aab4-21cc9adb4c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:03.100932 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59596d-3998-47f6-aab4-21cc9adb4c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:03.101331 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59596d-3998-47f6-aab4-21cc9adb4c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:03.101693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59596d-3998-47f6-aab4-21cc9adb4c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:03.102488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59596d-3998-47f6-aab4-21cc9adb4c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:03.103383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6b77d-d86e-454d-b4b6-c2781470ccb1 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:03.105418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59596d-3998-47f6-aab4-21cc9adb4c27 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:03.130541 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:03.140886 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6b77d-d86e-454d-b4b6-c2781470ccb1 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:03.142500 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 939/1909] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:03.143514 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59596d-3998-47f6-aab4-21cc9adb4c27 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:03.149460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eabc2-5cbe-499c-ade1-0625d1edf7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:03.150290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eabc2-5cbe-499c-ade1-0625d1edf7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:03.155704 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eabc2-5cbe-499c-ade1-0625d1edf7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:03.156214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eabc2-5cbe-499c-ade1-0625d1edf7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:03.159026 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29eabc2-5cbe-499c-ade1-0625d1edf7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:03.160537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59596d-3998-47f6-aab4-21cc9adb4c27 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:03.161886 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29eabc2-5cbe-499c-ade1-0625d1edf7c6 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:03.166730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db6505-9c91-46a2-971a-a25ecda048cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:03.167976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db6505-9c91-46a2-971a-a25ecda048cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:03.168361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db6505-9c91-46a2-971a-a25ecda048cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:03.168663 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db6505-9c91-46a2-971a-a25ecda048cb admin admin] PATH_INFO: `/v3/projects/0241c4524ca54b11abc9c7fe825de844/users/06feda7ec27e48c087e4e0e467092235/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:03.196511 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 971/1910] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:03.197996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eabc2-5cbe-499c-ade1-0625d1edf7c6 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:03.211212 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db6505-9c91-46a2-971a-a25ecda048cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0241c4524ca54b11abc9c7fe825de844, user_id=06feda7ec27e48c087e4e0e467092235, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:03.212714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eabc2-5cbe-499c-ade1-0625d1edf7c6 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:03.220224 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf3f34-bb36-4007-b5de-ceef2db4df2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:03.220761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db6505-9c91-46a2-971a-a25ecda048cb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:03.220930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf3f34-bb36-4007-b5de-ceef2db4df2a admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:03.221217 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf3f34-bb36-4007-b5de-ceef2db4df2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:03.221495 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf3f34-bb36-4007-b5de-ceef2db4df2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:03.224601 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf3f34-bb36-4007-b5de-ceef2db4df2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:03.228177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf3f34-bb36-4007-b5de-ceef2db4df2a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:03.262376 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 972/1911] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:03.263184 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 940/1912] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:03 2022] PUT /identity/v3/projects/0241c4524ca54b11abc9c7fe825de844/users/06feda7ec27e48c087e4e0e467092235/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:03.263738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf3f34-bb36-4007-b5de-ceef2db4df2a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:03.265873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db6505-9c91-46a2-971a-a25ecda048cb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:03.272888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7bcf3f34-bb36-4007-b5de-ceef2db4df2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:03.372140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db6505-9c91-46a2-971a-a25ecda048cb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:03.380192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4cbab-e0a2-488c-b56b-da22c831e5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:03.382747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4cbab-e0a2-488c-b56b-da22c831e5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:03.383114 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4cbab-e0a2-488c-b56b-da22c831e5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:03.383406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4cbab-e0a2-488c-b56b-da22c831e5a9 admin admin] PATH_INFO: `/v3/projects/7d8af7b6506641d7b06d4f6a55470651/users/c21be1766ab44458a876a9aae04ff8b0/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:03.403539 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf3f34-bb36-4007-b5de-ceef2db4df2a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:03.436236 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4cbab-e0a2-488c-b56b-da22c831e5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d8af7b6506641d7b06d4f6a55470651, user_id=c21be1766ab44458a876a9aae04ff8b0, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:03.443252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4cbab-e0a2-488c-b56b-da22c831e5a9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:03.484651 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 941/1913] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:03 2022] PUT /identity/v3/projects/7d8af7b6506641d7b06d4f6a55470651/users/c21be1766ab44458a876a9aae04ff8b0/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:03.484651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903676f-7db6-4729-bf9e-e5497a6f2aaf None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:03.484651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903676f-7db6-4729-bf9e-e5497a6f2aaf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:03.484651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903676f-7db6-4729-bf9e-e5497a6f2aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:03.485339 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93098607-7ba8-4001-aafe-7ab0d9467a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:03.485560 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93098607-7ba8-4001-aafe-7ab0d9467a28 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:03.486019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93098607-7ba8-4001-aafe-7ab0d9467a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:03.486344 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93098607-7ba8-4001-aafe-7ab0d9467a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:03.503657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-93098607-7ba8-4001-aafe-7ab0d9467a28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:03.550541 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:03.553184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d903676f-7db6-4729-bf9e-e5497a6f2aaf None None] MFA Rules not processed for user `06feda7ec27e48c087e4e0e467092235`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:03.556546 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d903676f-7db6-4729-bf9e-e5497a6f2aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:03.625800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93098607-7ba8-4001-aafe-7ab0d9467a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:03.627370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93098607-7ba8-4001-aafe-7ab0d9467a28 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:03.638044 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 973/1914] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:03.640774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06ee85-6abb-47ce-897d-4a382cf3fba9 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:03.641082 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06ee85-6abb-47ce-897d-4a382cf3fba9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:03.641780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06ee85-6abb-47ce-897d-4a382cf3fba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:03.672122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-9a06ee85-6abb-47ce-897d-4a382cf3fba9 None None] MFA Rules not processed for user `c21be1766ab44458a876a9aae04ff8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:03.674062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9a06ee85-6abb-47ce-897d-4a382cf3fba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:03.801254 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 974/1915] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:03.801426 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 942/1916] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:03.806537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06ee85-6abb-47ce-897d-4a382cf3fba9 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:03.867452 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903676f-7db6-4729-bf9e-e5497a6f2aaf None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:03.894452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06ee85-6abb-47ce-897d-4a382cf3fba9 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:03.899800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad868e-71ae-4f89-a47f-5228ec22c8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:03.900513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad868e-71ae-4f89-a47f-5228ec22c8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:03.900804 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad868e-71ae-4f89-a47f-5228ec22c8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:03.901104 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad868e-71ae-4f89-a47f-5228ec22c8c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:03.901820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad868e-71ae-4f89-a47f-5228ec22c8c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:03.904496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad868e-71ae-4f89-a47f-5228ec22c8c5 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:03.960253 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 975/1917] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:03 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:03.964589 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad868e-71ae-4f89-a47f-5228ec22c8c5 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:03.995731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:03.997996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903676f-7db6-4729-bf9e-e5497a6f2aaf None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.021554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:04.052045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad868e-71ae-4f89-a47f-5228ec22c8c5 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.057494 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ec097-b58f-4ca1-a4aa-e8a8eaa9e4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.058036 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ec097-b58f-4ca1-a4aa-e8a8eaa9e4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.058705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ec097-b58f-4ca1-a4aa-e8a8eaa9e4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.058988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ec097-b58f-4ca1-a4aa-e8a8eaa9e4e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.059752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ec097-b58f-4ca1-a4aa-e8a8eaa9e4e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.062510 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ec097-b58f-4ca1-a4aa-e8a8eaa9e4e3 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.086817 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 976/1918] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:03 2022] POST /identity/v3/users => generated 298 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:04.091590 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ec097-b58f-4ca1-a4aa-e8a8eaa9e4e3 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:04.105096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ec097-b58f-4ca1-a4aa-e8a8eaa9e4e3 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.119159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c7f68-ecdf-45de-b7da-42f10def7de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.126467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7f68-ecdf-45de-b7da-42f10def7de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.127000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7f68-ecdf-45de-b7da-42f10def7de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.127299 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7f68-ecdf-45de-b7da-42f10def7de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.132579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c7f68-ecdf-45de-b7da-42f10def7de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.138254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c7f68-ecdf-45de-b7da-42f10def7de7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.158951 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 977/1919] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:04.167435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70241d2a-edea-4b50-8d4e-4e1b34aad01e None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.167821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70241d2a-edea-4b50-8d4e-4e1b34aad01e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.168136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70241d2a-edea-4b50-8d4e-4e1b34aad01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.190873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc38537-b9d7-4f58-82d2-7f65c3c9a228 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.193787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc38537-b9d7-4f58-82d2-7f65c3c9a228 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.193995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc38537-b9d7-4f58-82d2-7f65c3c9a228 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.194268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc38537-b9d7-4f58-82d2-7f65c3c9a228 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.222161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-70241d2a-edea-4b50-8d4e-4e1b34aad01e None None] MFA Rules not processed for user `c323490c11b9461898687e729cd357f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:04.223421 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-70241d2a-edea-4b50-8d4e-4e1b34aad01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:04.294051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc38537-b9d7-4f58-82d2-7f65c3c9a228 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.296118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc38537-b9d7-4f58-82d2-7f65c3c9a228 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.308327 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 978/1920] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:04.309543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70241d2a-edea-4b50-8d4e-4e1b34aad01e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:04.310694 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 943/1921] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:04.312399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc38537-b9d7-4f58-82d2-7f65c3c9a228 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:04.330914 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc38537-b9d7-4f58-82d2-7f65c3c9a228 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.335338 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70241d2a-edea-4b50-8d4e-4e1b34aad01e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.337789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ef5cd-49c7-473a-a5e8-f45362d1f511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.338685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef5cd-49c7-473a-a5e8-f45362d1f511 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.339074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef5cd-49c7-473a-a5e8-f45362d1f511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.339456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef5cd-49c7-473a-a5e8-f45362d1f511 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.341149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ef5cd-49c7-473a-a5e8-f45362d1f511 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.342269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a3666-4da8-4cc0-adf1-7bb3f68209d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.343591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a3666-4da8-4cc0-adf1-7bb3f68209d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.344132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a3666-4da8-4cc0-adf1-7bb3f68209d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.344235 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ef5cd-49c7-473a-a5e8-f45362d1f511 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.344703 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a3666-4da8-4cc0-adf1-7bb3f68209d4 admin admin] PATH_INFO: `/v3/projects/3e1c6f0b6fde4d238400e388a73ac33a/users/9b030cc19de649028e84c85d2cd5e8b8/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.361726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a3666-4da8-4cc0-adf1-7bb3f68209d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e1c6f0b6fde4d238400e388a73ac33a, user_id=9b030cc19de649028e84c85d2cd5e8b8, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.365115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a3666-4da8-4cc0-adf1-7bb3f68209d4 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.394056 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 979/1922] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:04 2022] PUT /identity/v3/projects/3e1c6f0b6fde4d238400e388a73ac33a/users/9b030cc19de649028e84c85d2cd5e8b8/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:04.394281 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 944/1923] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:04 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:04.395301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a3666-4da8-4cc0-adf1-7bb3f68209d4 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:04.398080 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ef5cd-49c7-473a-a5e8-f45362d1f511 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:04.483990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ef5cd-49c7-473a-a5e8-f45362d1f511 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.486128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a3666-4da8-4cc0-adf1-7bb3f68209d4 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.490560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762aa106-4f35-4a9e-8df7-6c11bad2084c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.491714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762aa106-4f35-4a9e-8df7-6c11bad2084c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.492247 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762aa106-4f35-4a9e-8df7-6c11bad2084c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.492941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2249029-151d-44e2-84f3-98f4341bfb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.493498 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762aa106-4f35-4a9e-8df7-6c11bad2084c admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.494308 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2249029-151d-44e2-84f3-98f4341bfb23 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.494623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2249029-151d-44e2-84f3-98f4341bfb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.494906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2249029-151d-44e2-84f3-98f4341bfb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.495243 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762aa106-4f35-4a9e-8df7-6c11bad2084c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.496103 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2249029-151d-44e2-84f3-98f4341bfb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.498135 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762aa106-4f35-4a9e-8df7-6c11bad2084c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.498503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2249029-151d-44e2-84f3-98f4341bfb23 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.514257 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 980/1924] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:04.516226 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2249029-151d-44e2-84f3-98f4341bfb23 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:04.533190 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2249029-151d-44e2-84f3-98f4341bfb23 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.537697 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 945/1925] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:04 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:04.539376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762aa106-4f35-4a9e-8df7-6c11bad2084c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:04.542605 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c146727-0503-4435-8b4f-2e0fe18b10f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.544127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c146727-0503-4435-8b4f-2e0fe18b10f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.544631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c146727-0503-4435-8b4f-2e0fe18b10f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.545340 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c146727-0503-4435-8b4f-2e0fe18b10f4 admin admin] PATH_INFO: `/v3/projects/3e1c6f0b6fde4d238400e388a73ac33a/users/9b030cc19de649028e84c85d2cd5e8b8/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.547994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c146727-0503-4435-8b4f-2e0fe18b10f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e1c6f0b6fde4d238400e388a73ac33a, user_id=9b030cc19de649028e84c85d2cd5e8b8, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.551183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c146727-0503-4435-8b4f-2e0fe18b10f4 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.551383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762aa106-4f35-4a9e-8df7-6c11bad2084c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.558857 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762bd0b-4b5e-41a4-be1e-ecb8d902318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.560010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762bd0b-4b5e-41a4-be1e-ecb8d902318d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.560568 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762bd0b-4b5e-41a4-be1e-ecb8d902318d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.561102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762bd0b-4b5e-41a4-be1e-ecb8d902318d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.562010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762bd0b-4b5e-41a4-be1e-ecb8d902318d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.565096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762bd0b-4b5e-41a4-be1e-ecb8d902318d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.577336 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 981/1926] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:04 2022] PUT /identity/v3/projects/3e1c6f0b6fde4d238400e388a73ac33a/users/9b030cc19de649028e84c85d2cd5e8b8/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:04.582272 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 946/1927] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:04.586019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c146727-0503-4435-8b4f-2e0fe18b10f4 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:04.587859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addc4e0-4a11-4c03-998a-cbb56385d9d5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.588336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addc4e0-4a11-4c03-998a-cbb56385d9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.588755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addc4e0-4a11-4c03-998a-cbb56385d9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.599313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:04.622104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-6addc4e0-4a11-4c03-998a-cbb56385d9d5 None None] MFA Rules not processed for user `9b030cc19de649028e84c85d2cd5e8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:04.623876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6addc4e0-4a11-4c03-998a-cbb56385d9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:04.628450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:04.681323 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c146727-0503-4435-8b4f-2e0fe18b10f4 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.687629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07229c7a-1157-4d7b-ac13-27b6cd0e322f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.689147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229c7a-1157-4d7b-ac13-27b6cd0e322f admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.689390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229c7a-1157-4d7b-ac13-27b6cd0e322f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.689710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229c7a-1157-4d7b-ac13-27b6cd0e322f admin admin] PATH_INFO: `/v3/projects/fd9c63344ebc46eead81ad9d7269e239/users/1c639857118347fea2e5927efaaa492f/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.710364 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 947/1928] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:04.711528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07229c7a-1157-4d7b-ac13-27b6cd0e322f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9c63344ebc46eead81ad9d7269e239, user_id=1c639857118347fea2e5927efaaa492f, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.714529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07229c7a-1157-4d7b-ac13-27b6cd0e322f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.720794 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addc4e0-4a11-4c03-998a-cbb56385d9d5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:04.730813 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 982/1929] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:04 2022] PUT /identity/v3/projects/fd9c63344ebc46eead81ad9d7269e239/users/1c639857118347fea2e5927efaaa492f/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:04.734777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07229c7a-1157-4d7b-ac13-27b6cd0e322f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:04.784755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addc4e0-4a11-4c03-998a-cbb56385d9d5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.808376 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07229c7a-1157-4d7b-ac13-27b6cd0e322f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.814494 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f32228-9426-437b-b165-3de717fd9f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.815112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f32228-9426-437b-b165-3de717fd9f12 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.815396 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f32228-9426-437b-b165-3de717fd9f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.815677 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f32228-9426-437b-b165-3de717fd9f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.816417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f32228-9426-437b-b165-3de717fd9f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.818041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f32228-9426-437b-b165-3de717fd9f12 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.831865 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 983/1930] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:04.833190 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f32228-9426-437b-b165-3de717fd9f12 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:04.837430 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c85e8a-c935-443c-9042-c8ae65b1ab19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.838163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c85e8a-c935-443c-9042-c8ae65b1ab19 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.838456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c85e8a-c935-443c-9042-c8ae65b1ab19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.838738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c85e8a-c935-443c-9042-c8ae65b1ab19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.843553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f32228-9426-437b-b165-3de717fd9f12 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:04.848940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814a0c7-4df4-443f-9bb7-42dc961a09b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:04.849820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814a0c7-4df4-443f-9bb7-42dc961a09b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:04.850102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814a0c7-4df4-443f-9bb7-42dc961a09b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:04.850412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814a0c7-4df4-443f-9bb7-42dc961a09b5 admin admin] PATH_INFO: `/v3/projects/fd9c63344ebc46eead81ad9d7269e239/users/1c639857118347fea2e5927efaaa492f/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:04.854949 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814a0c7-4df4-443f-9bb7-42dc961a09b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9c63344ebc46eead81ad9d7269e239, user_id=1c639857118347fea2e5927efaaa492f, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.857708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814a0c7-4df4-443f-9bb7-42dc961a09b5 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.870467 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 984/1931] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:04 2022] PUT /identity/v3/projects/fd9c63344ebc46eead81ad9d7269e239/users/1c639857118347fea2e5927efaaa492f/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:04.915527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c85e8a-c935-443c-9042-c8ae65b1ab19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:04.917399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c85e8a-c935-443c-9042-c8ae65b1ab19 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:04.934033 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 948/1932] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:05.035334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814a0c7-4df4-443f-9bb7-42dc961a09b5 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:05.137560 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814a0c7-4df4-443f-9bb7-42dc961a09b5 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:05.144436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:05.145840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:05.146271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:05.146592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] PATH_INFO: `/v3/users/06feda7ec27e48c087e4e0e467092235` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:05.149483 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06feda7ec27e48c087e4e0e467092235)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:05.152534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:05.173244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:05.239650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06feda7ec27e48c087e4e0e467092235'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:05.240378 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06feda7ec27e48c087e4e0e467092235'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:05.258706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06feda7ec27e48c087e4e0e467092235'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:05.274839 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06feda7ec27e48c087e4e0e467092235'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:05.277255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06feda7ec27e48c087e4e0e467092235'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:05.325321 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 985/1933] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:05 2022] DELETE /identity/v3/users/06feda7ec27e48c087e4e0e467092235 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:05.330096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463935d6-cac0-4df3-ba07-17adf8037b6d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:05.331413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463935d6-cac0-4df3-ba07-17adf8037b6d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:05.332046 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463935d6-cac0-4df3-ba07-17adf8037b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:05.360437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:05.380875 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-463935d6-cac0-4df3-ba07-17adf8037b6d None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:05.382118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-463935d6-cac0-4df3-ba07-17adf8037b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:05.409306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:05.412293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:05.513995 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 949/1934] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:05.636015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd94485-ba6c-44bb-b6b9-910f524633f1 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:05.649525 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463935d6-cac0-4df3-ba07-17adf8037b6d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:05.650969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-463935d6-cac0-4df3-ba07-17adf8037b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:05.653325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13295522-d2b6-4936-a47e-70baa5e56012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:05.654961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13295522-d2b6-4936-a47e-70baa5e56012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:05.655361 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13295522-d2b6-4936-a47e-70baa5e56012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:05.655747 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13295522-d2b6-4936-a47e-70baa5e56012 admin admin] PATH_INFO: `/v3/users/adf6b9d16af549a6be1ffdd6cfe0fd6f` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:05.658009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13295522-d2b6-4936-a47e-70baa5e56012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adf6b9d16af549a6be1ffdd6cfe0fd6f)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:05.665954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13295522-d2b6-4936-a47e-70baa5e56012 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:05.779080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-13295522-d2b6-4936-a47e-70baa5e56012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adf6b9d16af549a6be1ffdd6cfe0fd6f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:05.779880 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-13295522-d2b6-4936-a47e-70baa5e56012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adf6b9d16af549a6be1ffdd6cfe0fd6f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:05.805091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-13295522-d2b6-4936-a47e-70baa5e56012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adf6b9d16af549a6be1ffdd6cfe0fd6f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:05.830962 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-13295522-d2b6-4936-a47e-70baa5e56012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adf6b9d16af549a6be1ffdd6cfe0fd6f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:05.831937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-13295522-d2b6-4936-a47e-70baa5e56012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adf6b9d16af549a6be1ffdd6cfe0fd6f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:05.853046 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 986/1935] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:05 2022] DELETE /identity/v3/users/adf6b9d16af549a6be1ffdd6cfe0fd6f => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:05.858510 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872224f-a280-41bd-a79c-7d1d1ac0a98a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:05.858808 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872224f-a280-41bd-a79c-7d1d1ac0a98a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:05.859097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872224f-a280-41bd-a79c-7d1d1ac0a98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:05.875220 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463935d6-cac0-4df3-ba07-17adf8037b6d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:05.875993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:05.892316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a872224f-a280-41bd-a79c-7d1d1ac0a98a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:05.893780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a872224f-a280-41bd-a79c-7d1d1ac0a98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:05.997882 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 987/1936] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:06.024115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67700f06-c773-4050-a472-7988af7480af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:06.024819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67700f06-c773-4050-a472-7988af7480af service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:06.025107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67700f06-c773-4050-a472-7988af7480af service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:06.025392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67700f06-c773-4050-a472-7988af7480af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:06.032232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-67700f06-c773-4050-a472-7988af7480af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:06.057578 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c1b18-da18-4a68-890f-c639a66e22d3 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:06.058057 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c1b18-da18-4a68-890f-c639a66e22d3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:06.058250 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c1b18-da18-4a68-890f-c639a66e22d3 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:06.060810 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 988/1937] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:13:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 10:13:06.067164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67700f06-c773-4050-a472-7988af7480af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:06.069105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67700f06-c773-4050-a472-7988af7480af service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:06.088019 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 950/1938] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:06.108231 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084276cb-94f9-4872-80c2-6a9d7bfcf495 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:06.108526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084276cb-94f9-4872-80c2-6a9d7bfcf495 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:06.108832 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084276cb-94f9-4872-80c2-6a9d7bfcf495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:06.140678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67700f06-c773-4050-a472-7988af7480af service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:06.142425 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-67700f06-c773-4050-a472-7988af7480af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:06.168055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-084276cb-94f9-4872-80c2-6a9d7bfcf495 None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:06.168256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67700f06-c773-4050-a472-7988af7480af service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:06.169821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-084276cb-94f9-4872-80c2-6a9d7bfcf495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:06.186704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6392b-b285-4ac2-ad37-f5b894e2a3fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:06.187399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6392b-b285-4ac2-ad37-f5b894e2a3fc service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:06.190512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6392b-b285-4ac2-ad37-f5b894e2a3fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:06.191897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6392b-b285-4ac2-ad37-f5b894e2a3fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:06.226816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6392b-b285-4ac2-ad37-f5b894e2a3fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:06.229086 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6392b-b285-4ac2-ad37-f5b894e2a3fc service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:06.248257 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 951/1939] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:06.298812 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 989/1940] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:13:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:06.325072 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6392b-b285-4ac2-ad37-f5b894e2a3fc service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:06.326561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b0e6392b-b285-4ac2-ad37-f5b894e2a3fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:06.344351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6392b-b285-4ac2-ad37-f5b894e2a3fc service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:06.351643 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084b557-5373-4dfb-9e06-3cf51a32ab43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:06.352318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084b557-5373-4dfb-9e06-3cf51a32ab43 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:06.352605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084b557-5373-4dfb-9e06-3cf51a32ab43 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:06.352878 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084b557-5373-4dfb-9e06-3cf51a32ab43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:06.363241 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084b557-5373-4dfb-9e06-3cf51a32ab43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:06.364984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084b557-5373-4dfb-9e06-3cf51a32ab43 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:06.377038 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 952/1941] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:06.650409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4ccfa-d90f-41fe-b827-f1db08a3675f None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:06.650704 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4ccfa-d90f-41fe-b827-f1db08a3675f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:06.650989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4ccfa-d90f-41fe-b827-f1db08a3675f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:06.736335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-efe4ccfa-d90f-41fe-b827-f1db08a3675f None None] MFA Rules not processed for user `1c639857118347fea2e5927efaaa492f`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:06.737595 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-efe4ccfa-d90f-41fe-b827-f1db08a3675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:06.956895 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 990/1942] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:06.961181 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad3d00-8e01-4445-ad88-c013b6f54f54 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:06.961475 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad3d00-8e01-4445-ad88-c013b6f54f54 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:06.961799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad3d00-8e01-4445-ad88-c013b6f54f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:06.999723 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-88ad3d00-8e01-4445-ad88-c013b6f54f54 None None] MFA Rules not processed for user `c323490c11b9461898687e729cd357f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:07.002188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-88ad3d00-8e01-4445-ad88-c013b6f54f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:07.059115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:07.099927 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 953/1943] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:07.109761 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4ccfa-d90f-41fe-b827-f1db08a3675f None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:07.111929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-efe4ccfa-d90f-41fe-b827-f1db08a3675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:07.133804 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4ccfa-d90f-41fe-b827-f1db08a3675f None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:07.142810 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a0e24-da8f-4498-af76-5a00a54ebbec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:07.143379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0e24-da8f-4498-af76-5a00a54ebbec service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:07.143775 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0e24-da8f-4498-af76-5a00a54ebbec service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:07.144347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0e24-da8f-4498-af76-5a00a54ebbec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:07.146201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-527a0e24-da8f-4498-af76-5a00a54ebbec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:07.160975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a0e24-da8f-4498-af76-5a00a54ebbec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:07.164324 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a0e24-da8f-4498-af76-5a00a54ebbec service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:07.177318 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 991/1944] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:07.599782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad3d00-8e01-4445-ad88-c013b6f54f54 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:07.601433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-88ad3d00-8e01-4445-ad88-c013b6f54f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:07.635146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad3d00-8e01-4445-ad88-c013b6f54f54 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:07.649991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20696109-672c-4199-bb79-1910552186ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:07.653359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20696109-672c-4199-bb79-1910552186ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:07.655263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20696109-672c-4199-bb79-1910552186ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:07.663476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20696109-672c-4199-bb79-1910552186ed admin admin] PATH_INFO: `/v3/projects/0241c4524ca54b11abc9c7fe825de844` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:07.664803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20696109-672c-4199-bb79-1910552186ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=0241c4524ca54b11abc9c7fe825de844)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:07.666879 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20696109-672c-4199-bb79-1910552186ed admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:07.781244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20696109-672c-4199-bb79-1910552186ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0241c4524ca54b11abc9c7fe825de844', 'request_id': 'req-20696109-672c-4199-bb79-1910552186ed'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:07.800273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20696109-672c-4199-bb79-1910552186ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0241c4524ca54b11abc9c7fe825de844', 'request_id': 'req-20696109-672c-4199-bb79-1910552186ed'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:07.813890 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20696109-672c-4199-bb79-1910552186ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0241c4524ca54b11abc9c7fe825de844', 'request_id': 'req-20696109-672c-4199-bb79-1910552186ed'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:07.855481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20696109-672c-4199-bb79-1910552186ed admin admin] The token cache is being invalidate because project 0241c4524ca54b11abc9c7fe825de844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:07.857234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20696109-672c-4199-bb79-1910552186ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:07.858675 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-20696109-672c-4199-bb79-1910552186ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:07.864743 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 954/1945] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:07 2022] DELETE /identity/v3/projects/0241c4524ca54b11abc9c7fe825de844 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:07.868971 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a0e24-da8f-4498-af76-5a00a54ebbec service nova] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:07.888120 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a0e24-da8f-4498-af76-5a00a54ebbec service nova] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:07.890273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20696109-672c-4199-bb79-1910552186ed admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:07.891954 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-20696109-672c-4199-bb79-1910552186ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:07.922350 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:08.049607 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:08.050557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:08.050843 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:08.051124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] PATH_INFO: `/v3/users/9b030cc19de649028e84c85d2cd5e8b8` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:08.055069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b030cc19de649028e84c85d2cd5e8b8)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:08.058656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:08.060010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20696109-672c-4199-bb79-1910552186ed admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:08.069452 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:08.070384 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:08.070819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:08.071101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] PATH_INFO: `/v3/projects/a0b57d468b3a464fb436cb9168c97199` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:08.072382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b57d468b3a464fb436cb9168c97199)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:08.075523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:08.190799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b57d468b3a464fb436cb9168c97199', 'request_id': 'req-d956c2c1-803c-4425-bf0b-c81e7e104db3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.204406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b030cc19de649028e84c85d2cd5e8b8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.204950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b57d468b3a464fb436cb9168c97199', 'request_id': 'req-d956c2c1-803c-4425-bf0b-c81e7e104db3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.206556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b030cc19de649028e84c85d2cd5e8b8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.217476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b57d468b3a464fb436cb9168c97199', 'request_id': 'req-d956c2c1-803c-4425-bf0b-c81e7e104db3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.233901 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:08.240497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] The token cache is being invalidate because project a0b57d468b3a464fb436cb9168c97199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:08.241768 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.244076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.246926 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 955/1946] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:07 2022] DELETE /identity/v3/projects/a0b57d468b3a464fb436cb9168c97199 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:08.253659 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:08.256353 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:08.258370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b030cc19de649028e84c85d2cd5e8b8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.271688 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b030cc19de649028e84c85d2cd5e8b8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.274543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b030cc19de649028e84c85d2cd5e8b8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.299028 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 992/1947] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:07 2022] DELETE /identity/v3/users/9b030cc19de649028e84c85d2cd5e8b8 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:08.386808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c2c1-803c-4425-bf0b-c81e7e104db3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:08.519058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:08.521454 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:08.523730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:08.524765 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] PATH_INFO: `/v3/users/2689d792e7a34ad9bee81d5698252153` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:08.529786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2689d792e7a34ad9bee81d5698252153)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:08.535938 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:08.699783 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2689d792e7a34ad9bee81d5698252153'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.701272 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2689d792e7a34ad9bee81d5698252153'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.702433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2689d792e7a34ad9bee81d5698252153'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.726021 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2689d792e7a34ad9bee81d5698252153'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.740647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2689d792e7a34ad9bee81d5698252153'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:08.803468 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 956/1948] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:08 2022] DELETE /identity/v3/users/2689d792e7a34ad9bee81d5698252153 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:09.583827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:09.585845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:09.604467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892ebdb-e72f-492a-93c1-ab32455fc575 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:09.712537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:09.718700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:09.723273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:09.724543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:09.724971 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:09.725398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] PATH_INFO: `/v3/projects/3e1c6f0b6fde4d238400e388a73ac33a` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:09.727073 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e1c6f0b6fde4d238400e388a73ac33a)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:09.739330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:09.743210 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc04a2-60a2-4308-a458-26b5020fe38b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:09.771911 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:09.772890 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:09.773196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:09.773460 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] PATH_INFO: `/v3/projects/abe3db42c8434f9bb254dc338b293417` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:09.777103 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe3db42c8434f9bb254dc338b293417)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:09.786161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:09.820760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:09.846741 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:09.849111 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e1c6f0b6fde4d238400e388a73ac33a', 'request_id': 'req-0e7fe897-406f-47c8-a988-b964ca45d8a8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:09.861502 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e1c6f0b6fde4d238400e388a73ac33a', 'request_id': 'req-0e7fe897-406f-47c8-a988-b964ca45d8a8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:09.880329 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e1c6f0b6fde4d238400e388a73ac33a', 'request_id': 'req-0e7fe897-406f-47c8-a988-b964ca45d8a8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:09.894432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe3db42c8434f9bb254dc338b293417', 'request_id': 'req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:09.904164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] The token cache is being invalidate because project 3e1c6f0b6fde4d238400e388a73ac33a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:09.904562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:09.905479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-0e7fe897-406f-47c8-a988-b964ca45d8a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:09.906945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe3db42c8434f9bb254dc338b293417', 'request_id': 'req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:09.913892 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 993/1949] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:09 2022] DELETE /identity/v3/projects/3e1c6f0b6fde4d238400e388a73ac33a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:09.915841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aedef-b891-4ed8-bcba-11921f4eac20 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:09.916042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe3db42c8434f9bb254dc338b293417', 'request_id': 'req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:09.916212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aedef-b891-4ed8-bcba-11921f4eac20 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:09.916591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aedef-b891-4ed8-bcba-11921f4eac20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:09.936119 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] The token cache is being invalidate because project abe3db42c8434f9bb254dc338b293417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:09.936409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:09.937053 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-078b2cb7-4f11-43f9-8b5a-d9b15f3082a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:09.937988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-8e0aedef-b891-4ed8-bcba-11921f4eac20 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:09.938659 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 957/1950] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:09 2022] DELETE /identity/v3/projects/abe3db42c8434f9bb254dc338b293417 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:09.942488 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8e0aedef-b891-4ed8-bcba-11921f4eac20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:09.944455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33795e-be3f-4ef6-bc7f-7de519b7ca30 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:09.944873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33795e-be3f-4ef6-bc7f-7de519b7ca30 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:09.945209 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33795e-be3f-4ef6-bc7f-7de519b7ca30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:09.967453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ad33795e-be3f-4ef6-bc7f-7de519b7ca30 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:09.968762 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ad33795e-be3f-4ef6-bc7f-7de519b7ca30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.019850 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 994/1951] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:10.020713 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0aedef-b891-4ed8-bcba-11921f4eac20 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.021970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8e0aedef-b891-4ed8-bcba-11921f4eac20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.032209 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 958/1952] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:10.033794 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bf051-2592-43d1-be63-58cfe5dd534a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.034082 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bf051-2592-43d1-be63-58cfe5dd534a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.034389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bf051-2592-43d1-be63-58cfe5dd534a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.040607 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0aedef-b891-4ed8-bcba-11921f4eac20 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.045489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.046934 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.047363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.047638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] PATH_INFO: `/v3/users/f14703de0a894961911ccb34ad1036b4` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.048868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f14703de0a894961911ccb34ad1036b4)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.050909 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.055923 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-676bf051-2592-43d1-be63-58cfe5dd534a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:10.057226 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-676bf051-2592-43d1-be63-58cfe5dd534a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.066526 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 959/1953] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:10.067922 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f627e74-468d-484f-b008-09156ba9e414 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.068280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f627e74-468d-484f-b008-09156ba9e414 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.068612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f627e74-468d-484f-b008-09156ba9e414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.088918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8f627e74-468d-484f-b008-09156ba9e414 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:10.091144 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8f627e74-468d-484f-b008-09156ba9e414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.104582 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 960/1954] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:10.106727 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f627e74-468d-484f-b008-09156ba9e414 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.117826 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f627e74-468d-484f-b008-09156ba9e414 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.119771 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f14703de0a894961911ccb34ad1036b4'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:10.120911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f14703de0a894961911ccb34ad1036b4'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:10.123363 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09004463-beab-4fe0-9542-3ba56a992f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.124058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09004463-beab-4fe0-9542-3ba56a992f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.124338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09004463-beab-4fe0-9542-3ba56a992f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.124618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09004463-beab-4fe0-9542-3ba56a992f8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.125600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09004463-beab-4fe0-9542-3ba56a992f8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:10.125972 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09004463-beab-4fe0-9542-3ba56a992f8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.127867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09004463-beab-4fe0-9542-3ba56a992f8d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.137680 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f14703de0a894961911ccb34ad1036b4'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:10.144426 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 961/1955] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:10.144426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09004463-beab-4fe0-9542-3ba56a992f8d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.145201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-09004463-beab-4fe0-9542-3ba56a992f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.145452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f14703de0a894961911ccb34ad1036b4'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:10.146319 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f14703de0a894961911ccb34ad1036b4'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:10.163269 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09004463-beab-4fe0-9542-3ba56a992f8d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.167880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567a7bf-2be3-47ab-b12f-b582b4edeeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.168014 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 995/1956] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:10 2022] DELETE /identity/v3/users/f14703de0a894961911ccb34ad1036b4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:10.168790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567a7bf-2be3-47ab-b12f-b582b4edeeae admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.168897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.169077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567a7bf-2be3-47ab-b12f-b582b4edeeae admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.169351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567a7bf-2be3-47ab-b12f-b582b4edeeae admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.170105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567a7bf-2be3-47ab-b12f-b582b4edeeae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:10.170440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567a7bf-2be3-47ab-b12f-b582b4edeeae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.172086 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.172229 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567a7bf-2be3-47ab-b12f-b582b4edeeae admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.185814 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 962/1957] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:10.187163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd34ded-74d6-4c66-a6e9-fa2222aec3b7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.187459 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd34ded-74d6-4c66-a6e9-fa2222aec3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.188875 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd34ded-74d6-4c66-a6e9-fa2222aec3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.208904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3dd34ded-74d6-4c66-a6e9-fa2222aec3b7 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:10.210626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3dd34ded-74d6-4c66-a6e9-fa2222aec3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.224497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:10.271945 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b66ce3-a3f9-4db1-9a90-fd64b2d9609c admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.276841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472352c6-bc58-442a-bbb1-427265691ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.277519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472352c6-bc58-442a-bbb1-427265691ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.277795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472352c6-bc58-442a-bbb1-427265691ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.278065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472352c6-bc58-442a-bbb1-427265691ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.278781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472352c6-bc58-442a-bbb1-427265691ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.280684 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472352c6-bc58-442a-bbb1-427265691ad6 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.288026 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 963/1958] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:10.289011 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd34ded-74d6-4c66-a6e9-fa2222aec3b7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.299242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd34ded-74d6-4c66-a6e9-fa2222aec3b7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.303904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdc7c0-1d50-4e07-b1d4-90f37e47c89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.304566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc7c0-1d50-4e07-b1d4-90f37e47c89e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.304841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc7c0-1d50-4e07-b1d4-90f37e47c89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.305121 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc7c0-1d50-4e07-b1d4-90f37e47c89e admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.305822 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bdc7c0-1d50-4e07-b1d4-90f37e47c89e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.310806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bdc7c0-1d50-4e07-b1d4-90f37e47c89e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.310948 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 996/1959] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:10 2022] POST /identity/v3/projects => generated 399 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:10.310948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472352c6-bc58-442a-bbb1-427265691ad6 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.310948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-472352c6-bc58-442a-bbb1-427265691ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.312790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:10.344847 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 964/1960] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:10 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:10.344847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdc7c0-1d50-4e07-b1d4-90f37e47c89e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.368376 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472352c6-bc58-442a-bbb1-427265691ad6 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.412574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdc7c0-1d50-4e07-b1d4-90f37e47c89e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.417610 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf64757-004b-477f-b281-06ef79fcc1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.418441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf64757-004b-477f-b281-06ef79fcc1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.418792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf64757-004b-477f-b281-06ef79fcc1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.419138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf64757-004b-477f-b281-06ef79fcc1f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.419848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf64757-004b-477f-b281-06ef79fcc1f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.421782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf64757-004b-477f-b281-06ef79fcc1f3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.435554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dfe82-0e77-447f-9d5a-1fe825fb4dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.436165 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dfe82-0e77-447f-9d5a-1fe825fb4dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.436490 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dfe82-0e77-447f-9d5a-1fe825fb4dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.436765 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dfe82-0e77-447f-9d5a-1fe825fb4dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.446284 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 965/1961] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:10 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:10.447190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf64757-004b-477f-b281-06ef79fcc1f3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.450481 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dfe82-0e77-447f-9d5a-1fe825fb4dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.452156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dfe82-0e77-447f-9d5a-1fe825fb4dc9 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.457332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf64757-004b-477f-b281-06ef79fcc1f3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.462211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a7d0-c9bc-463e-b085-0a8d06101ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.462566 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 997/1962] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:10.462900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a7d0-c9bc-463e-b085-0a8d06101ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.463181 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a7d0-c9bc-463e-b085-0a8d06101ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.463440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dfe82-0e77-447f-9d5a-1fe825fb4dc9 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.463546 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a7d0-c9bc-463e-b085-0a8d06101ec8 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.464163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98a7d0-c9bc-463e-b085-0a8d06101ec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.466069 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98a7d0-c9bc-463e-b085-0a8d06101ec8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.473233 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dfe82-0e77-447f-9d5a-1fe825fb4dc9 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.478337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b305c-9135-4273-bdf6-5cefde94fc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.479288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b305c-9135-4273-bdf6-5cefde94fc61 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.479649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b305c-9135-4273-bdf6-5cefde94fc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.480028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b305c-9135-4273-bdf6-5cefde94fc61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.481031 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b305c-9135-4273-bdf6-5cefde94fc61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.483054 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b305c-9135-4273-bdf6-5cefde94fc61 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.490490 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 966/1963] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:10 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:10.491734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a7d0-c9bc-463e-b085-0a8d06101ec8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.499731 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 998/1964] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:10.500806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b305c-9135-4273-bdf6-5cefde94fc61 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.505142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a7d0-c9bc-463e-b085-0a8d06101ec8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.509927 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8396e3-eb5e-416f-9ea1-6413f1016cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.510648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8396e3-eb5e-416f-9ea1-6413f1016cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.511109 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8396e3-eb5e-416f-9ea1-6413f1016cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.511280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8396e3-eb5e-416f-9ea1-6413f1016cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.512075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8396e3-eb5e-416f-9ea1-6413f1016cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.513142 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b305c-9135-4273-bdf6-5cefde94fc61 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.513857 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8396e3-eb5e-416f-9ea1-6413f1016cd9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.520136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08bb1a-4723-4125-8b56-65ebd7f31431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.520901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08bb1a-4723-4125-8b56-65ebd7f31431 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.521247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08bb1a-4723-4125-8b56-65ebd7f31431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.521862 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08bb1a-4723-4125-8b56-65ebd7f31431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.522961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08bb1a-4723-4125-8b56-65ebd7f31431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.524721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08bb1a-4723-4125-8b56-65ebd7f31431 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.528910 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 967/1965] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:10.531469 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8396e3-eb5e-416f-9ea1-6413f1016cd9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.536163 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 999/1966] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:10.541465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08bb1a-4723-4125-8b56-65ebd7f31431 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.547542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8396e3-eb5e-416f-9ea1-6413f1016cd9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.553531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15988a70-60e7-48d7-8408-c3d1475d92a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.554358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15988a70-60e7-48d7-8408-c3d1475d92a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.555055 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15988a70-60e7-48d7-8408-c3d1475d92a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.555207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08bb1a-4723-4125-8b56-65ebd7f31431 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.555451 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15988a70-60e7-48d7-8408-c3d1475d92a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.556249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15988a70-60e7-48d7-8408-c3d1475d92a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.557949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15988a70-60e7-48d7-8408-c3d1475d92a7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.560206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cf051-7539-468a-9572-45ec99ee57e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.561310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cf051-7539-468a-9572-45ec99ee57e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.562015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cf051-7539-468a-9572-45ec99ee57e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.562015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cf051-7539-468a-9572-45ec99ee57e2 admin admin] PATH_INFO: `/v3/projects/7ee670637eec4eda9a7b6462cc5b5069/users/5311020be0ee491c810919e3665b613e/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.568781 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 968/1967] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:10.572776 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15988a70-60e7-48d7-8408-c3d1475d92a7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.574575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-15988a70-60e7-48d7-8408-c3d1475d92a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.578507 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cf051-7539-468a-9572-45ec99ee57e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee670637eec4eda9a7b6462cc5b5069, user_id=5311020be0ee491c810919e3665b613e, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.581492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cf051-7539-468a-9572-45ec99ee57e2 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.589193 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15988a70-60e7-48d7-8408-c3d1475d92a7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.600858 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26042705-b6e2-44d9-8f41-bc8d11482d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.603083 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26042705-b6e2-44d9-8f41-bc8d11482d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.603622 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26042705-b6e2-44d9-8f41-bc8d11482d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.604585 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26042705-b6e2-44d9-8f41-bc8d11482d38 admin admin] PATH_INFO: `/v3/users/c21be1766ab44458a876a9aae04ff8b0` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.606154 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1000/1968] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:10 2022] PUT /identity/v3/projects/7ee670637eec4eda9a7b6462cc5b5069/users/5311020be0ee491c810919e3665b613e/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:10.615303 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cf051-7539-468a-9572-45ec99ee57e2 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.615516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26042705-b6e2-44d9-8f41-bc8d11482d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c21be1766ab44458a876a9aae04ff8b0)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.615516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26042705-b6e2-44d9-8f41-bc8d11482d38 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.650500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:10.686802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cf051-7539-468a-9572-45ec99ee57e2 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:10.693160 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ee1a6-b736-4f96-b76d-d9b1f4d888bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:10.694208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee1a6-b736-4f96-b76d-d9b1f4d888bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.694521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee1a6-b736-4f96-b76d-d9b1f4d888bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.694844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee1a6-b736-4f96-b76d-d9b1f4d888bc admin admin] PATH_INFO: `/v3/projects/df0575c2dee64315851eac5f73862fa6/users/fb18afcd78c34b4d84b00f79020664c7/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.697713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-26042705-b6e2-44d9-8f41-bc8d11482d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c21be1766ab44458a876a9aae04ff8b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:10.700756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-26042705-b6e2-44d9-8f41-bc8d11482d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c21be1766ab44458a876a9aae04ff8b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:10.701582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-26042705-b6e2-44d9-8f41-bc8d11482d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c21be1766ab44458a876a9aae04ff8b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:10.713228 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-26042705-b6e2-44d9-8f41-bc8d11482d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c21be1766ab44458a876a9aae04ff8b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:10.713850 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ee1a6-b736-4f96-b76d-d9b1f4d888bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0575c2dee64315851eac5f73862fa6, user_id=fb18afcd78c34b4d84b00f79020664c7, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:10.716322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ee1a6-b736-4f96-b76d-d9b1f4d888bc admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:10.729626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-26042705-b6e2-44d9-8f41-bc8d11482d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c21be1766ab44458a876a9aae04ff8b0'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:10.746474 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1001/1969] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:10 2022] PUT /identity/v3/projects/df0575c2dee64315851eac5f73862fa6/users/fb18afcd78c34b4d84b00f79020664c7/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:10.746474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8256489-cfec-4979-885d-947b678d4fc3 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.746474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8256489-cfec-4979-885d-947b678d4fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.746474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8256489-cfec-4979-885d-947b678d4fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.764871 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 969/1970] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:10 2022] DELETE /identity/v3/users/c21be1766ab44458a876a9aae04ff8b0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:10.766392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba967f3f-c139-4e8c-a774-97159037e723 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.769257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba967f3f-c139-4e8c-a774-97159037e723 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.769633 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba967f3f-c139-4e8c-a774-97159037e723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.775809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-f8256489-cfec-4979-885d-947b678d4fc3 None None] MFA Rules not processed for user `5311020be0ee491c810919e3665b613e`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:10.777738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f8256489-cfec-4979-885d-947b678d4fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.807984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=82,keystone:INSERT=8,keystone:DELETE=10 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:10.829800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ba967f3f-c139-4e8c-a774-97159037e723 None None] MFA Rules not processed for user `fb18afcd78c34b4d84b00f79020664c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:10.831393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ba967f3f-c139-4e8c-a774-97159037e723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.891417 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1002/1971] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:10.892587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b514e9-a596-49eb-97bb-586b2c3ebc69 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:10.892881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b514e9-a596-49eb-97bb-586b2c3ebc69 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:10.893167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b514e9-a596-49eb-97bb-586b2c3ebc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:10.922485 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-10b514e9-a596-49eb-97bb-586b2c3ebc69 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:10.923140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-10b514e9-a596-49eb-97bb-586b2c3ebc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:10.930256 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 970/1972] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:10.931328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba967f3f-c139-4e8c-a774-97159037e723 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:10.932701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ba967f3f-c139-4e8c-a774-97159037e723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:11.041976 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1003/1973] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:11.044444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b514e9-a596-49eb-97bb-586b2c3ebc69 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.051017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-10b514e9-a596-49eb-97bb-586b2c3ebc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:11.064570 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba967f3f-c139-4e8c-a774-97159037e723 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.069158 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b514e9-a596-49eb-97bb-586b2c3ebc69 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.069488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.071212 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.072071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.072365 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] PATH_INFO: `/v3/projects/7a9fa441b8864879bef5e4d29f1b7fef` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.073891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9fa441b8864879bef5e4d29f1b7fef)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.076377 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeba247-2b58-4480-88d9-d2a8a2f9692b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.076590 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.076960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeba247-2b58-4480-88d9-d2a8a2f9692b admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.077356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeba247-2b58-4480-88d9-d2a8a2f9692b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.089023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeba247-2b58-4480-88d9-d2a8a2f9692b admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.090014 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeba247-2b58-4480-88d9-d2a8a2f9692b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.096689 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeba247-2b58-4480-88d9-d2a8a2f9692b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.147947 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1004/1974] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:11 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:11.149158 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeba247-2b58-4480-88d9-d2a8a2f9692b admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.151176 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8aeba247-2b58-4480-88d9-d2a8a2f9692b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:11.156251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9fa441b8864879bef5e4d29f1b7fef', 'request_id': 'req-39bd4e84-ab51-43db-9701-eff870da65ca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:11.165394 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9fa441b8864879bef5e4d29f1b7fef', 'request_id': 'req-39bd4e84-ab51-43db-9701-eff870da65ca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:11.175426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9fa441b8864879bef5e4d29f1b7fef', 'request_id': 'req-39bd4e84-ab51-43db-9701-eff870da65ca'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:11.188076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:11.195672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] The token cache is being invalidate because project 7a9fa441b8864879bef5e4d29f1b7fef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:11.196054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:11.197125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:11.201812 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 971/1975] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:10 2022] DELETE /identity/v3/projects/7a9fa441b8864879bef5e4d29f1b7fef => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:11.201812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.202062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:11.249094 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeba247-2b58-4480-88d9-d2a8a2f9692b admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.249754 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:11.256506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce7247-b54f-4b3c-90e6-b0331a7d1073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.257420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7247-b54f-4b3c-90e6-b0331a7d1073 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.257807 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7247-b54f-4b3c-90e6-b0331a7d1073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.258151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7247-b54f-4b3c-90e6-b0331a7d1073 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.258970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce7247-b54f-4b3c-90e6-b0331a7d1073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.261579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce7247-b54f-4b3c-90e6-b0331a7d1073 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.288121 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd4e84-ab51-43db-9701-eff870da65ca admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.302587 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1005/1976] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:11 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:11.304167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce7247-b54f-4b3c-90e6-b0331a7d1073 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.307259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-18ce7247-b54f-4b3c-90e6-b0331a7d1073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:11.352884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b7936-2958-45df-8e0e-ed613f6568db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.353659 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b7936-2958-45df-8e0e-ed613f6568db service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.354043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b7936-2958-45df-8e0e-ed613f6568db service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.354453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b7936-2958-45df-8e0e-ed613f6568db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.356487 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a55b7936-2958-45df-8e0e-ed613f6568db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:11.369481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b7936-2958-45df-8e0e-ed613f6568db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.371107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b7936-2958-45df-8e0e-ed613f6568db service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.388460 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 972/1977] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:11.389359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b7936-2958-45df-8e0e-ed613f6568db service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.390725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce7247-b54f-4b3c-90e6-b0331a7d1073 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.391001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a55b7936-2958-45df-8e0e-ed613f6568db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:11.395945 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efde23-f844-410f-9c13-0ee0cbff0d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.396804 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efde23-f844-410f-9c13-0ee0cbff0d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.397344 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efde23-f844-410f-9c13-0ee0cbff0d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.397623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efde23-f844-410f-9c13-0ee0cbff0d3a admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.398384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efde23-f844-410f-9c13-0ee0cbff0d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.400486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efde23-f844-410f-9c13-0ee0cbff0d3a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.403753 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b7936-2958-45df-8e0e-ed613f6568db service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.408560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.409559 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.410019 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.410395 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] PATH_INFO: `/v3/users/c937dc53cc7a411badb3e494efa52680` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.411730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c937dc53cc7a411badb3e494efa52680)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.413832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.429005 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1006/1978] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:11 2022] POST /identity/v3/users => generated 307 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:11.429005 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efde23-f844-410f-9c13-0ee0cbff0d3a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.431022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-16efde23-f844-410f-9c13-0ee0cbff0d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:11.443254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efde23-f844-410f-9c13-0ee0cbff0d3a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.447961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ac857-f9e3-4c63-857f-30ebf36ad86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.448728 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ac857-f9e3-4c63-857f-30ebf36ad86a admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.449002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ac857-f9e3-4c63-857f-30ebf36ad86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.449352 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ac857-f9e3-4c63-857f-30ebf36ad86a admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.450122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ac857-f9e3-4c63-857f-30ebf36ad86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.452489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ac857-f9e3-4c63-857f-30ebf36ad86a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.475882 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1007/1979] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:11 2022] POST /identity/v3/users => generated 296 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:11.476722 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ac857-f9e3-4c63-857f-30ebf36ad86a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.482913 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c937dc53cc7a411badb3e494efa52680'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:11.483719 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c937dc53cc7a411badb3e494efa52680'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:11.485042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c937dc53cc7a411badb3e494efa52680'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:11.488791 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ac857-f9e3-4c63-857f-30ebf36ad86a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.494784 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974d3a9-268f-4138-a368-ac7c7a60a926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.495660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974d3a9-268f-4138-a368-ac7c7a60a926 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.496042 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974d3a9-268f-4138-a368-ac7c7a60a926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.496587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974d3a9-268f-4138-a368-ac7c7a60a926 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.498078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974d3a9-268f-4138-a368-ac7c7a60a926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.500069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974d3a9-268f-4138-a368-ac7c7a60a926 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.505323 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c937dc53cc7a411badb3e494efa52680'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:11.514819 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1008/1980] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:11.515769 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974d3a9-268f-4138-a368-ac7c7a60a926 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.519390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b974d3a9-268f-4138-a368-ac7c7a60a926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:11.534707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974d3a9-268f-4138-a368-ac7c7a60a926 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.535484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c937dc53cc7a411badb3e494efa52680'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:11.540950 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01a07c-b626-4695-9072-afb026b097bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.541839 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a07c-b626-4695-9072-afb026b097bd admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.542342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a07c-b626-4695-9072-afb026b097bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.542725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01a07c-b626-4695-9072-afb026b097bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.543718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01a07c-b626-4695-9072-afb026b097bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.545951 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01a07c-b626-4695-9072-afb026b097bd admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.561441 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1009/1981] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:11.562527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01a07c-b626-4695-9072-afb026b097bd admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.565546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ed01a07c-b626-4695-9072-afb026b097bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:11.583876 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 973/1982] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:11 2022] DELETE /identity/v3/users/c937dc53cc7a411badb3e494efa52680 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:11.585718 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.606694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01a07c-b626-4695-9072-afb026b097bd admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.661996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dafe5-c3f8-4dd8-8186-94b9f059b6fa admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.667352 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e9321-ac42-491e-95c5-0173eaaf160c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.668513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e9321-ac42-491e-95c5-0173eaaf160c admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.669473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e9321-ac42-491e-95c5-0173eaaf160c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.669873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e9321-ac42-491e-95c5-0173eaaf160c admin admin] PATH_INFO: `/v3/projects/0333c3e7e7a947909970b8857630955c/users/71e6664b05ee44d18790aef2b4e480f1/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.672621 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:11.690777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e9321-ac42-491e-95c5-0173eaaf160c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0333c3e7e7a947909970b8857630955c, user_id=71e6664b05ee44d18790aef2b4e480f1, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.694517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e9321-ac42-491e-95c5-0173eaaf160c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.718122 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 974/1983] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:11 2022] PUT /identity/v3/projects/0333c3e7e7a947909970b8857630955c/users/71e6664b05ee44d18790aef2b4e480f1/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:11.722267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e9321-ac42-491e-95c5-0173eaaf160c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.733238 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:11.741676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9aece-ec28-4c90-8ca6-a48833274292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.743118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9aece-ec28-4c90-8ca6-a48833274292 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.743479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9aece-ec28-4c90-8ca6-a48833274292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.744204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9aece-ec28-4c90-8ca6-a48833274292 admin admin] PATH_INFO: `/v3/projects/a17db58263f844af825f4d4c043b4ad0/users/0668c19227d847639574ab533a75e143/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.767237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9aece-ec28-4c90-8ca6-a48833274292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17db58263f844af825f4d4c043b4ad0, user_id=0668c19227d847639574ab533a75e143, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.771353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9aece-ec28-4c90-8ca6-a48833274292 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.791196 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1010/1984] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:11 2022] PUT /identity/v3/projects/a17db58263f844af825f4d4c043b4ad0/users/0668c19227d847639574ab533a75e143/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:11.794928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9aece-ec28-4c90-8ca6-a48833274292 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.825715 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e9321-ac42-491e-95c5-0173eaaf160c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.878723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9aece-ec28-4c90-8ca6-a48833274292 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.885571 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c876f-de8e-484c-91a0-d108ba4a080f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.886743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c876f-de8e-484c-91a0-d108ba4a080f admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.887373 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c876f-de8e-484c-91a0-d108ba4a080f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.887749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c876f-de8e-484c-91a0-d108ba4a080f admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.888822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c876f-de8e-484c-91a0-d108ba4a080f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.891131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c876f-de8e-484c-91a0-d108ba4a080f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.904387 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1011/1985] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:11.909360 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c876f-de8e-484c-91a0-d108ba4a080f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.926939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c876f-de8e-484c-91a0-d108ba4a080f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.932625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90597b-a197-41b2-bbb6-19b3c308c94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.933554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90597b-a197-41b2-bbb6-19b3c308c94d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.933910 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90597b-a197-41b2-bbb6-19b3c308c94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.934311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90597b-a197-41b2-bbb6-19b3c308c94d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.934812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df5afa-d492-4e11-81f3-28eda6b70655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:11.935643 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90597b-a197-41b2-bbb6-19b3c308c94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.936802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df5afa-d492-4e11-81f3-28eda6b70655 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.937262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df5afa-d492-4e11-81f3-28eda6b70655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.937487 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90597b-a197-41b2-bbb6-19b3c308c94d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.938023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df5afa-d492-4e11-81f3-28eda6b70655 admin admin] PATH_INFO: `/v3/projects/a17db58263f844af825f4d4c043b4ad0/users/0668c19227d847639574ab533a75e143/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:11.940497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df5afa-d492-4e11-81f3-28eda6b70655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17db58263f844af825f4d4c043b4ad0, user_id=0668c19227d847639574ab533a75e143, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:11.943703 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df5afa-d492-4e11-81f3-28eda6b70655 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:11.949870 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 975/1986] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:11.953934 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90597b-a197-41b2-bbb6-19b3c308c94d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:11.966107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90597b-a197-41b2-bbb6-19b3c308c94d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:11.978325 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1012/1987] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:11 2022] PUT /identity/v3/projects/a17db58263f844af825f4d4c043b4ad0/users/0668c19227d847639574ab533a75e143/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:11.987215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723618-e1e9-4be1-8433-59814092e92d None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:11.987507 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723618-e1e9-4be1-8433-59814092e92d None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:11.987792 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723618-e1e9-4be1-8433-59814092e92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.038171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-23723618-e1e9-4be1-8433-59814092e92d None None] MFA Rules not processed for user `0668c19227d847639574ab533a75e143`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:12.040126 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-23723618-e1e9-4be1-8433-59814092e92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.042800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e1b9f-61bd-40f7-92ab-1eb9abee885f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:12.044036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e1b9f-61bd-40f7-92ab-1eb9abee885f admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.044468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e1b9f-61bd-40f7-92ab-1eb9abee885f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.045211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e1b9f-61bd-40f7-92ab-1eb9abee885f admin admin] PATH_INFO: `/v3/projects/0333c3e7e7a947909970b8857630955c/users/71e6664b05ee44d18790aef2b4e480f1/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.049822 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e1b9f-61bd-40f7-92ab-1eb9abee885f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0333c3e7e7a947909970b8857630955c, user_id=71e6664b05ee44d18790aef2b4e480f1, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:12.053335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e1b9f-61bd-40f7-92ab-1eb9abee885f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:12.067927 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 976/1988] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:11 2022] PUT /identity/v3/projects/0333c3e7e7a947909970b8857630955c/users/71e6664b05ee44d18790aef2b4e480f1/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:12.068930 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e1b9f-61bd-40f7-92ab-1eb9abee885f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:12.084757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-492e1b9f-61bd-40f7-92ab-1eb9abee885f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.117736 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:12.165320 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1013/1989] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:11 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:12.165320 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54e953-3e00-488c-8a50-fc51e733c961 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.165320 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54e953-3e00-488c-8a50-fc51e733c961 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.165320 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54e953-3e00-488c-8a50-fc51e733c961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.200631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-8c54e953-3e00-488c-8a50-fc51e733c961 None None] MFA Rules not processed for user `71e6664b05ee44d18790aef2b4e480f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:12.201149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e1b9f-61bd-40f7-92ab-1eb9abee885f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:12.202102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8c54e953-3e00-488c-8a50-fc51e733c961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.214683 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:12.218616 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.221186 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.221503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] PATH_INFO: `/v3/projects/b2531e238d6b4c26800c638330b83acf` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.227125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2531e238d6b4c26800c638330b83acf)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:12.234579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:12.278658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:12.329297 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2531e238d6b4c26800c638330b83acf', 'request_id': 'req-65f50c68-53e9-49f8-b78e-07dddcf4626c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:12.331329 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1014/1990] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:12.332540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54e953-3e00-488c-8a50-fc51e733c961 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:12.335013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8c54e953-3e00-488c-8a50-fc51e733c961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.341903 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2531e238d6b4c26800c638330b83acf', 'request_id': 'req-65f50c68-53e9-49f8-b78e-07dddcf4626c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:12.354991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2531e238d6b4c26800c638330b83acf', 'request_id': 'req-65f50c68-53e9-49f8-b78e-07dddcf4626c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:12.367160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] The token cache is being invalidate because project b2531e238d6b4c26800c638330b83acf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:12.367702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:12.368379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:12.370986 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 977/1991] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:12 2022] DELETE /identity/v3/projects/b2531e238d6b4c26800c638330b83acf => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:12.371739 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:12.376567 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.431138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54e953-3e00-488c-8a50-fc51e733c961 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:12.471315 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f50c68-53e9-49f8-b78e-07dddcf4626c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:12.478425 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:12.479598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.480008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.480347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] PATH_INFO: `/v3/projects/7d8af7b6506641d7b06d4f6a55470651` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.484361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d8af7b6506641d7b06d4f6a55470651)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:12.486329 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:12.511482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e6cf2-7f71-42e2-a7c4-1dc10dc9b6f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:12.512658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e6cf2-7f71-42e2-a7c4-1dc10dc9b6f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.513155 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e6cf2-7f71-42e2-a7c4-1dc10dc9b6f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.513510 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e6cf2-7f71-42e2-a7c4-1dc10dc9b6f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.515565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8a9e6cf2-7f71-42e2-a7c4-1dc10dc9b6f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.548815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d8af7b6506641d7b06d4f6a55470651', 'request_id': 'req-04fd8a1a-ef24-49b7-849e-e241f2d38942'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:12.552486 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:12.559838 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d8af7b6506641d7b06d4f6a55470651', 'request_id': 'req-04fd8a1a-ef24-49b7-849e-e241f2d38942'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:12.568028 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d8af7b6506641d7b06d4f6a55470651', 'request_id': 'req-04fd8a1a-ef24-49b7-849e-e241f2d38942'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:12.581415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] The token cache is being invalidate because project 7d8af7b6506641d7b06d4f6a55470651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:12.581714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:12.582429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:12.588138 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 978/1992] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:12 2022] DELETE /identity/v3/projects/7d8af7b6506641d7b06d4f6a55470651 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:12.588138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:12.588138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.591278 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e6cf2-7f71-42e2-a7c4-1dc10dc9b6f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:12.593159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e6cf2-7f71-42e2-a7c4-1dc10dc9b6f4 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:12.607906 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1015/1993] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:12 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:12.610748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d405e8-e62a-4b18-b71c-5b6af924bdfc None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.611390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d405e8-e62a-4b18-b71c-5b6af924bdfc None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.611955 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d405e8-e62a-4b18-b71c-5b6af924bdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.637264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-13d405e8-e62a-4b18-b71c-5b6af924bdfc None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:12.639094 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-13d405e8-e62a-4b18-b71c-5b6af924bdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.681059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:12.715001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd8a1a-ef24-49b7-849e-e241f2d38942 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:12.724708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b437e6-6672-489a-8665-2d989b052295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:12.727399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b437e6-6672-489a-8665-2d989b052295 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.727734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b437e6-6672-489a-8665-2d989b052295 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.728784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b437e6-6672-489a-8665-2d989b052295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.734440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-96b437e6-6672-489a-8665-2d989b052295 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.739001 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1016/1994] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:12.742081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad69a2-89f7-4b15-ae44-e21079eabdfa None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.742395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad69a2-89f7-4b15-ae44-e21079eabdfa None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.742676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad69a2-89f7-4b15-ae44-e21079eabdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.766972 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-16ad69a2-89f7-4b15-ae44-e21079eabdfa None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:12.770369 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-16ad69a2-89f7-4b15-ae44-e21079eabdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.787172 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1017/1995] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:12.788153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad69a2-89f7-4b15-ae44-e21079eabdfa None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:12.789615 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-16ad69a2-89f7-4b15-ae44-e21079eabdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.807026 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad69a2-89f7-4b15-ae44-e21079eabdfa None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:12.812723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d59c81-d732-4505-a6c9-4c217f1cc577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:12.813405 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59c81-d732-4505-a6c9-4c217f1cc577 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.813689 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59c81-d732-4505-a6c9-4c217f1cc577 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.813966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59c81-d732-4505-a6c9-4c217f1cc577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.816142 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-26d59c81-d732-4505-a6c9-4c217f1cc577 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.827432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b437e6-6672-489a-8665-2d989b052295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:12.829419 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b437e6-6672-489a-8665-2d989b052295 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:12.844172 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 979/1996] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:12.845937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50dad5-1375-47bd-b607-f9e2b59bf57c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.846374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50dad5-1375-47bd-b607-f9e2b59bf57c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.846958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50dad5-1375-47bd-b607-f9e2b59bf57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.877348 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9d50dad5-1375-47bd-b607-f9e2b59bf57c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:12.878895 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9d50dad5-1375-47bd-b607-f9e2b59bf57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.894371 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 980/1997] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:12.896900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300bd15-d4db-4e12-bc33-f1eb8840ba53 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.897570 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300bd15-d4db-4e12-bc33-f1eb8840ba53 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.898024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300bd15-d4db-4e12-bc33-f1eb8840ba53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.898450 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d59c81-d732-4505-a6c9-4c217f1cc577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:12.900834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d59c81-d732-4505-a6c9-4c217f1cc577 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:12.914080 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1018/1998] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:12.916004 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d59c81-d732-4505-a6c9-4c217f1cc577 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:12.921234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1300bd15-d4db-4e12-bc33-f1eb8840ba53 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:12.923079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1300bd15-d4db-4e12-bc33-f1eb8840ba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:12.929460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d59c81-d732-4505-a6c9-4c217f1cc577 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:12.937789 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 981/1999] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:12.938445 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a9700-6d6d-4569-924d-0541971f9671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:12.939768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a9700-6d6d-4569-924d-0541971f9671 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.940200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a9700-6d6d-4569-924d-0541971f9671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.940613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a9700-6d6d-4569-924d-0541971f9671 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.942932 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300bd15-d4db-4e12-bc33-f1eb8840ba53 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:12.945409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a9700-6d6d-4569-924d-0541971f9671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:12.945899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a9700-6d6d-4569-924d-0541971f9671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:12.947763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a9700-6d6d-4569-924d-0541971f9671 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:12.963982 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300bd15-d4db-4e12-bc33-f1eb8840ba53 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:12.966743 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1019/2000] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:12.968762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a9700-6d6d-4569-924d-0541971f9671 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:12.977518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fad89-0f68-4cab-a233-7bcfb4d5222a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:12.978213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fad89-0f68-4cab-a233-7bcfb4d5222a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.978518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fad89-0f68-4cab-a233-7bcfb4d5222a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.978794 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fad89-0f68-4cab-a233-7bcfb4d5222a admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.980066 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fad89-0f68-4cab-a233-7bcfb4d5222a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:12.980066 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fad89-0f68-4cab-a233-7bcfb4d5222a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:12.981670 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fad89-0f68-4cab-a233-7bcfb4d5222a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:12.987792 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a9700-6d6d-4569-924d-0541971f9671 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:12.993694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc64b40-6387-4f6e-8564-edbbb06727e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:12.994996 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 982/2001] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:12.996891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc64b40-6387-4f6e-8564-edbbb06727e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:12.997170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc64b40-6387-4f6e-8564-edbbb06727e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:12.997458 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc64b40-6387-4f6e-8564-edbbb06727e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:12.999875 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fad89-0f68-4cab-a233-7bcfb4d5222a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.002647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ccc64b40-6387-4f6e-8564-edbbb06727e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:13.010409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fad89-0f68-4cab-a233-7bcfb4d5222a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.015150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61a5b1-8a55-43b2-88af-191745ef58aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.015832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61a5b1-8a55-43b2-88af-191745ef58aa admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.016115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61a5b1-8a55-43b2-88af-191745ef58aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.016503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61a5b1-8a55-43b2-88af-191745ef58aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.017521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61a5b1-8a55-43b2-88af-191745ef58aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.019879 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61a5b1-8a55-43b2-88af-191745ef58aa admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.059338 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 983/2002] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:12 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:13.059743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61a5b1-8a55-43b2-88af-191745ef58aa admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.091643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc64b40-6387-4f6e-8564-edbbb06727e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.093826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc64b40-6387-4f6e-8564-edbbb06727e3 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.111244 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1020/2003] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:13.118803 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc64b40-6387-4f6e-8564-edbbb06727e3 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.139570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc64b40-6387-4f6e-8564-edbbb06727e3 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.168775 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:13.210808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61a5b1-8a55-43b2-88af-191745ef58aa admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.229162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c74c9-c06b-4e37-9646-73338453cd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.229899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c74c9-c06b-4e37-9646-73338453cd99 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.230187 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c74c9-c06b-4e37-9646-73338453cd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.230509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c74c9-c06b-4e37-9646-73338453cd99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.231223 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c74c9-c06b-4e37-9646-73338453cd99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.233291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c74c9-c06b-4e37-9646-73338453cd99 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.262328 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:13.273814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd157e97-fbe4-4a26-bb83-cf68c6778447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.274936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd157e97-fbe4-4a26-bb83-cf68c6778447 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.275436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd157e97-fbe4-4a26-bb83-cf68c6778447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.275738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd157e97-fbe4-4a26-bb83-cf68c6778447 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.276987 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd157e97-fbe4-4a26-bb83-cf68c6778447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.290527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd157e97-fbe4-4a26-bb83-cf68c6778447 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.294180 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 984/2004] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:13 2022] POST /identity/v3/projects => generated 375 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:13.296174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c74c9-c06b-4e37-9646-73338453cd99 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.337902 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1021/2005] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:13 2022] POST /identity/v3/users => generated 294 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:13.339000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd157e97-fbe4-4a26-bb83-cf68c6778447 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.352631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd157e97-fbe4-4a26-bb83-cf68c6778447 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.431076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c74c9-c06b-4e37-9646-73338453cd99 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.436625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f28f0a-ec44-4363-aa68-b44e047e2800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.437392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28f0a-ec44-4363-aa68-b44e047e2800 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.437748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28f0a-ec44-4363-aa68-b44e047e2800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.438096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28f0a-ec44-4363-aa68-b44e047e2800 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.438898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f28f0a-ec44-4363-aa68-b44e047e2800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.440908 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f28f0a-ec44-4363-aa68-b44e047e2800 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.471370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa1421-942c-4545-ba19-083466910bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.472617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1421-942c-4545-ba19-083466910bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.473280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1421-942c-4545-ba19-083466910bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.473672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1421-942c-4545-ba19-083466910bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.474547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa1421-942c-4545-ba19-083466910bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.476501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa1421-942c-4545-ba19-083466910bc8 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.484365 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 985/2006] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:13 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:13.488509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f28f0a-ec44-4363-aa68-b44e047e2800 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.490771 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1022/2007] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:13.494675 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa1421-942c-4545-ba19-083466910bc8 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.502282 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f28f0a-ec44-4363-aa68-b44e047e2800 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.508456 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa1421-942c-4545-ba19-083466910bc8 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.510266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d6972-8d8d-44a5-bbd5-4aaf24d1e88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.511382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d6972-8d8d-44a5-bbd5-4aaf24d1e88b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.511845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d6972-8d8d-44a5-bbd5-4aaf24d1e88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.512305 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d6972-8d8d-44a5-bbd5-4aaf24d1e88b admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.513209 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d6972-8d8d-44a5-bbd5-4aaf24d1e88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.515078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043a039-2f8c-48c7-87e2-6ea4b32ca42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.516109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043a039-2f8c-48c7-87e2-6ea4b32ca42a admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.516201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043a039-2f8c-48c7-87e2-6ea4b32ca42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.516475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043a039-2f8c-48c7-87e2-6ea4b32ca42a admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.516692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d6972-8d8d-44a5-bbd5-4aaf24d1e88b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.517302 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043a039-2f8c-48c7-87e2-6ea4b32ca42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.519257 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043a039-2f8c-48c7-87e2-6ea4b32ca42a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.532748 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1023/2008] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:13.534058 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 986/2009] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:13.536965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043a039-2f8c-48c7-87e2-6ea4b32ca42a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.538019 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d6972-8d8d-44a5-bbd5-4aaf24d1e88b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.549526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d6972-8d8d-44a5-bbd5-4aaf24d1e88b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.549875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043a039-2f8c-48c7-87e2-6ea4b32ca42a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.555025 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966b867-8437-493f-bcfd-d03e0e154f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.555851 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966b867-8437-493f-bcfd-d03e0e154f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.556059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05760d-2f76-45bc-bb14-fb08cb09252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.556176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966b867-8437-493f-bcfd-d03e0e154f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.556402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966b867-8437-493f-bcfd-d03e0e154f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.556970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05760d-2f76-45bc-bb14-fb08cb09252d admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.557137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966b867-8437-493f-bcfd-d03e0e154f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.557319 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05760d-2f76-45bc-bb14-fb08cb09252d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.557566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05760d-2f76-45bc-bb14-fb08cb09252d admin admin] PATH_INFO: `/v3/projects/3f13c9c831874e35815f7899f68e901f/users/ca264a249175410bb7597e2a69af2197/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.558928 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966b867-8437-493f-bcfd-d03e0e154f54 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.576917 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 987/2010] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:13.578714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966b867-8437-493f-bcfd-d03e0e154f54 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.579100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05760d-2f76-45bc-bb14-fb08cb09252d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f13c9c831874e35815f7899f68e901f, user_id=ca264a249175410bb7597e2a69af2197, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.582084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05760d-2f76-45bc-bb14-fb08cb09252d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.590697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966b867-8437-493f-bcfd-d03e0e154f54 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.597976 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1024/2011] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:13 2022] PUT /identity/v3/projects/3f13c9c831874e35815f7899f68e901f/users/ca264a249175410bb7597e2a69af2197/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:13.610931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8078cb0-7b01-417b-b720-47a3e89dcde5 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.611597 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8078cb0-7b01-417b-b720-47a3e89dcde5 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.611822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8078cb0-7b01-417b-b720-47a3e89dcde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.643091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b8078cb0-7b01-417b-b720-47a3e89dcde5 None None] MFA Rules not processed for user `ca264a249175410bb7597e2a69af2197`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:13.644360 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b8078cb0-7b01-417b-b720-47a3e89dcde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:13.677263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df590c6-c2d8-4f55-a879-d9b52aa0293d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.678487 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df590c6-c2d8-4f55-a879-d9b52aa0293d admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.679218 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df590c6-c2d8-4f55-a879-d9b52aa0293d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.679497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df590c6-c2d8-4f55-a879-d9b52aa0293d admin admin] PATH_INFO: `/v3/projects/3c105b455fba44b19c7d56f8e817130f/users/4c8d180b9dd2491fb4e5d8d500547734/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.698173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df590c6-c2d8-4f55-a879-d9b52aa0293d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c105b455fba44b19c7d56f8e817130f, user_id=4c8d180b9dd2491fb4e5d8d500547734, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.701345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df590c6-c2d8-4f55-a879-d9b52aa0293d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.721479 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 988/2012] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:13 2022] PUT /identity/v3/projects/3c105b455fba44b19c7d56f8e817130f/users/4c8d180b9dd2491fb4e5d8d500547734/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:13.729489 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1025/2013] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:13.733971 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8078cb0-7b01-417b-b720-47a3e89dcde5 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.734626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8ab5d-5964-4b41-a8ac-658a2a8c467c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.735125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8ab5d-5964-4b41-a8ac-658a2a8c467c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.735219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8ab5d-5964-4b41-a8ac-658a2a8c467c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.744876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8078cb0-7b01-417b-b720-47a3e89dcde5 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.767449 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f6b8ab5d-5964-4b41-a8ac-658a2a8c467c None None] MFA Rules not processed for user `4c8d180b9dd2491fb4e5d8d500547734`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:13.767914 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f6b8ab5d-5964-4b41-a8ac-658a2a8c467c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:13.774008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:13.787224 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:13.816247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c475db-49c7-46a9-b8b5-5adaba4fcd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.816928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c475db-49c7-46a9-b8b5-5adaba4fcd78 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.817206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c475db-49c7-46a9-b8b5-5adaba4fcd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.817478 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c475db-49c7-46a9-b8b5-5adaba4fcd78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.818169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c475db-49c7-46a9-b8b5-5adaba4fcd78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.820111 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c475db-49c7-46a9-b8b5-5adaba4fcd78 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.852637 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1026/2014] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:13 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:13.852944 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 989/2015] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:13.853663 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c475db-49c7-46a9-b8b5-5adaba4fcd78 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.854059 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8ab5d-5964-4b41-a8ac-658a2a8c467c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.865706 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8ab5d-5964-4b41-a8ac-658a2a8c467c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.919302 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c475db-49c7-46a9-b8b5-5adaba4fcd78 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.924177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02faec-4e74-4f9c-816f-0a991d4a6a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.924866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02faec-4e74-4f9c-816f-0a991d4a6a76 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.925288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02faec-4e74-4f9c-816f-0a991d4a6a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.925661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02faec-4e74-4f9c-816f-0a991d4a6a76 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.926613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02faec-4e74-4f9c-816f-0a991d4a6a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.928763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02faec-4e74-4f9c-816f-0a991d4a6a76 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.939470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5f22a-7c46-420b-adc1-bdd05b5e300f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.940591 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5f22a-7c46-420b-adc1-bdd05b5e300f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.941034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5f22a-7c46-420b-adc1-bdd05b5e300f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.941791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5f22a-7c46-420b-adc1-bdd05b5e300f admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.946139 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5f22a-7c46-420b-adc1-bdd05b5e300f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.948494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5f22a-7c46-420b-adc1-bdd05b5e300f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:13.961400 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1027/2016] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:13 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:13.965914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02faec-4e74-4f9c-816f-0a991d4a6a76 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.979955 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02faec-4e74-4f9c-816f-0a991d4a6a76 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:13.984647 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 990/2017] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:13 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:13.987232 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb53ca-54a6-494b-aafd-58fc8db8cfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:13.988079 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb53ca-54a6-494b-aafd-58fc8db8cfec admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:13.988448 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb53ca-54a6-494b-aafd-58fc8db8cfec admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:13.988788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb53ca-54a6-494b-aafd-58fc8db8cfec admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:13.989549 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb53ca-54a6-494b-aafd-58fc8db8cfec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:13.990052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5f22a-7c46-420b-adc1-bdd05b5e300f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:13.991258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb53ca-54a6-494b-aafd-58fc8db8cfec admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:14.002697 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1028/2018] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:14.006690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb53ca-54a6-494b-aafd-58fc8db8cfec admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:14.024246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb53ca-54a6-494b-aafd-58fc8db8cfec admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:14.060445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5f22a-7c46-420b-adc1-bdd05b5e300f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:14.074051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77c11f-6e4c-426a-8c98-8d4877f3c37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:14.075016 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77c11f-6e4c-426a-8c98-8d4877f3c37b admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:14.086449 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77c11f-6e4c-426a-8c98-8d4877f3c37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:14.086911 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77c11f-6e4c-426a-8c98-8d4877f3c37b admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:14.087800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77c11f-6e4c-426a-8c98-8d4877f3c37b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:14.090869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77c11f-6e4c-426a-8c98-8d4877f3c37b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:14.094960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a829ee-7261-41ea-9eb0-f3ddff5413b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:14.096488 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a829ee-7261-41ea-9eb0-f3ddff5413b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:14.096986 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a829ee-7261-41ea-9eb0-f3ddff5413b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:14.097491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a829ee-7261-41ea-9eb0-f3ddff5413b1 admin admin] PATH_INFO: `/v3/projects/4ff0ce01ffa4413f8282e2ae8e817b90/users/5be22decb9bf4b9d8cdad50ba1d4b21f/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:14.123929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a829ee-7261-41ea-9eb0-f3ddff5413b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff0ce01ffa4413f8282e2ae8e817b90, user_id=5be22decb9bf4b9d8cdad50ba1d4b21f, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:14.128574 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 991/2019] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:13 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:14.128717 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a829ee-7261-41ea-9eb0-f3ddff5413b1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:14.130245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77c11f-6e4c-426a-8c98-8d4877f3c37b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:14.142698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77c11f-6e4c-426a-8c98-8d4877f3c37b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:14.149338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916e52e-1cd0-4cba-bb10-1964eb99860a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:14.151307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916e52e-1cd0-4cba-bb10-1964eb99860a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:14.151307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916e52e-1cd0-4cba-bb10-1964eb99860a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:14.151307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916e52e-1cd0-4cba-bb10-1964eb99860a admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:14.151919 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916e52e-1cd0-4cba-bb10-1964eb99860a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:14.153649 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916e52e-1cd0-4cba-bb10-1964eb99860a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:14.163730 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1029/2020] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:14 2022] PUT /identity/v3/projects/4ff0ce01ffa4413f8282e2ae8e817b90/users/5be22decb9bf4b9d8cdad50ba1d4b21f/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:14.165156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a829ee-7261-41ea-9eb0-f3ddff5413b1 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:14.168726 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 992/2021] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:14.169782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916e52e-1cd0-4cba-bb10-1964eb99860a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:14.181375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916e52e-1cd0-4cba-bb10-1964eb99860a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:14.191302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:14.232838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:14.257706 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932c725-4ecc-4433-9843-bd524d3ea4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:14.258638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932c725-4ecc-4433-9843-bd524d3ea4dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:14.258918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932c725-4ecc-4433-9843-bd524d3ea4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:14.259193 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932c725-4ecc-4433-9843-bd524d3ea4dc admin admin] PATH_INFO: `/v3/projects/746e801af26d48ac943346f641ce9d90/users/0dcceded8b0f4a1bb4af81ee7ada8b47/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:14.278670 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932c725-4ecc-4433-9843-bd524d3ea4dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=746e801af26d48ac943346f641ce9d90, user_id=0dcceded8b0f4a1bb4af81ee7ada8b47, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:14.281628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932c725-4ecc-4433-9843-bd524d3ea4dc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:14.305518 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 993/2022] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:14 2022] PUT /identity/v3/projects/746e801af26d48ac943346f641ce9d90/users/0dcceded8b0f4a1bb4af81ee7ada8b47/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:14.310623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932c725-4ecc-4433-9843-bd524d3ea4dc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:14.312913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a829ee-7261-41ea-9eb0-f3ddff5413b1 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:14.400677 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932c725-4ecc-4433-9843-bd524d3ea4dc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:14.411997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887462a-e719-4f4e-8d24-b4bb659a60ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:14.414375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887462a-e719-4f4e-8d24-b4bb659a60ac admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:14.414583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887462a-e719-4f4e-8d24-b4bb659a60ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:14.414852 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887462a-e719-4f4e-8d24-b4bb659a60ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:14.417418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887462a-e719-4f4e-8d24-b4bb659a60ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:14.419112 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887462a-e719-4f4e-8d24-b4bb659a60ac admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:14.448245 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 994/2023] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:14.449268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887462a-e719-4f4e-8d24-b4bb659a60ac admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:14.471854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887462a-e719-4f4e-8d24-b4bb659a60ac admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:14.478025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0fca4-6b9d-4bd4-9e2a-fb761bd049e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:14.478710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0fca4-6b9d-4bd4-9e2a-fb761bd049e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:14.479005 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0fca4-6b9d-4bd4-9e2a-fb761bd049e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:14.479284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0fca4-6b9d-4bd4-9e2a-fb761bd049e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:14.480714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0fca4-6b9d-4bd4-9e2a-fb761bd049e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:14.485529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0fca4-6b9d-4bd4-9e2a-fb761bd049e0 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:14.488924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1faa52-a091-4bf6-b0f6-1ffb17e68af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:14.490024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1faa52-a091-4bf6-b0f6-1ffb17e68af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:14.490173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1faa52-a091-4bf6-b0f6-1ffb17e68af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:14.490552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1faa52-a091-4bf6-b0f6-1ffb17e68af3 admin admin] PATH_INFO: `/v3/projects/746e801af26d48ac943346f641ce9d90/users/0dcceded8b0f4a1bb4af81ee7ada8b47/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:14.499088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1faa52-a091-4bf6-b0f6-1ffb17e68af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=746e801af26d48ac943346f641ce9d90, user_id=0dcceded8b0f4a1bb4af81ee7ada8b47, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:14.501882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1faa52-a091-4bf6-b0f6-1ffb17e68af3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:14.522965 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1030/2024] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:14.527017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0fca4-6b9d-4bd4-9e2a-fb761bd049e0 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:14.532230 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 995/2025] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:14 2022] PUT /identity/v3/projects/746e801af26d48ac943346f641ce9d90/users/0dcceded8b0f4a1bb4af81ee7ada8b47/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:14.542030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217291-df3d-4ad5-aeee-f7c47f374275 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:14.542366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217291-df3d-4ad5-aeee-f7c47f374275 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:14.542649 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217291-df3d-4ad5-aeee-f7c47f374275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:14.550960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0fca4-6b9d-4bd4-9e2a-fb761bd049e0 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:14.572853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ed217291-df3d-4ad5-aeee-f7c47f374275 None None] MFA Rules not processed for user `0dcceded8b0f4a1bb4af81ee7ada8b47`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:14.574536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ed217291-df3d-4ad5-aeee-f7c47f374275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:14.673918 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 996/2026] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:14.689904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217291-df3d-4ad5-aeee-f7c47f374275 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:14.697756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:14.724332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbf8db-aad6-4480-ab64-4ff7bede9645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:14.724332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf8db-aad6-4480-ab64-4ff7bede9645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:14.724332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf8db-aad6-4480-ab64-4ff7bede9645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:14.724332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbf8db-aad6-4480-ab64-4ff7bede9645 admin admin] PATH_INFO: `/v3/projects/4ff0ce01ffa4413f8282e2ae8e817b90/users/5be22decb9bf4b9d8cdad50ba1d4b21f/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:14.726655 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbf8db-aad6-4480-ab64-4ff7bede9645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff0ce01ffa4413f8282e2ae8e817b90, user_id=5be22decb9bf4b9d8cdad50ba1d4b21f, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:14.729800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbf8db-aad6-4480-ab64-4ff7bede9645 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:14.745792 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1031/2027] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:14 2022] PUT /identity/v3/projects/4ff0ce01ffa4413f8282e2ae8e817b90/users/5be22decb9bf4b9d8cdad50ba1d4b21f/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:14.759022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2922e-c607-4caf-9766-47b98179bbd8 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:14.759022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2922e-c607-4caf-9766-47b98179bbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:14.759022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2922e-c607-4caf-9766-47b98179bbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:14.791916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217291-df3d-4ad5-aeee-f7c47f374275 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:14.803574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-c9c2922e-c607-4caf-9766-47b98179bbd8 None None] MFA Rules not processed for user `5be22decb9bf4b9d8cdad50ba1d4b21f`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:14.805135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c9c2922e-c607-4caf-9766-47b98179bbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:14.868202 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:14.881474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3718af-f979-4daa-8856-4892df328995 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:14.882149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3718af-f979-4daa-8856-4892df328995 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:14.882464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3718af-f979-4daa-8856-4892df328995 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:14.882746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3718af-f979-4daa-8856-4892df328995 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:14.891442 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1032/2028] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:15.002147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3718af-f979-4daa-8856-4892df328995 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:15.003691 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3718af-f979-4daa-8856-4892df328995 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:15.024535 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 997/2029] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:15.027532 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2922e-c607-4caf-9766-47b98179bbd8 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:15.038471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2922e-c607-4caf-9766-47b98179bbd8 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:15.044969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16022e7-484e-4380-b6ca-43fea3af81d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:15.046729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16022e7-484e-4380-b6ca-43fea3af81d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:15.047454 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16022e7-484e-4380-b6ca-43fea3af81d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:15.047800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16022e7-484e-4380-b6ca-43fea3af81d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:15.063277 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16022e7-484e-4380-b6ca-43fea3af81d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:15.065086 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16022e7-484e-4380-b6ca-43fea3af81d9 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:15.077282 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1033/2030] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:16.032879 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3718af-f979-4daa-8856-4892df328995 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:16.097059 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3718af-f979-4daa-8856-4892df328995 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:16.101475 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:16.102393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:16.102675 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:16.102948 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] PATH_INFO: `/v3/users/ca264a249175410bb7597e2a69af2197` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:16.104126 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca264a249175410bb7597e2a69af2197)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:16.106073 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:16.120194 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:16.156311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca264a249175410bb7597e2a69af2197'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.156968 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca264a249175410bb7597e2a69af2197'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.157552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca264a249175410bb7597e2a69af2197'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.164473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca264a249175410bb7597e2a69af2197'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.175882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca264a249175410bb7597e2a69af2197'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.188739 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 998/2031] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:16 2022] DELETE /identity/v3/users/ca264a249175410bb7597e2a69af2197 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:16.193228 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298afe97-568b-4966-8d7b-ac88cf671b77 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:16.193714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298afe97-568b-4966-8d7b-ac88cf671b77 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:16.193807 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298afe97-568b-4966-8d7b-ac88cf671b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:16.213152 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-298afe97-568b-4966-8d7b-ac88cf671b77 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:16.214501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-298afe97-568b-4966-8d7b-ac88cf671b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:16.277172 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1034/2032] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:16.293046 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:16.295246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:16.365760 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e6fff-e450-43a3-984e-1a70669de07e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:16.369872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ddfe1-2817-4cd1-b486-4b881f9d2ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:16.370632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ddfe1-2817-4cd1-b486-4b881f9d2ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:16.370919 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ddfe1-2817-4cd1-b486-4b881f9d2ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:16.371204 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ddfe1-2817-4cd1-b486-4b881f9d2ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:16.381480 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ddfe1-2817-4cd1-b486-4b881f9d2ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:16.383196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ddfe1-2817-4cd1-b486-4b881f9d2ac6 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:16.393748 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 999/2033] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:16.745722 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298afe97-568b-4966-8d7b-ac88cf671b77 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:16.747439 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-298afe97-568b-4966-8d7b-ac88cf671b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:16.772609 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298afe97-568b-4966-8d7b-ac88cf671b77 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:16.777469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:16.781138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:16.781138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:16.781138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] PATH_INFO: `/v3/projects/3f13c9c831874e35815f7899f68e901f` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:16.781138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f13c9c831874e35815f7899f68e901f)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:16.782050 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:16.838037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f13c9c831874e35815f7899f68e901f', 'request_id': 'req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.843944 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f13c9c831874e35815f7899f68e901f', 'request_id': 'req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.850309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f13c9c831874e35815f7899f68e901f', 'request_id': 'req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.862889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] The token cache is being invalidate because project 3f13c9c831874e35815f7899f68e901f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:16.863197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.863890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.865619 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1035/2034] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:16 2022] DELETE /identity/v3/projects/3f13c9c831874e35815f7899f68e901f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:16.869818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ddfe1-2817-4cd1-b486-4b881f9d2ac6 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:16.879650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ddfe1-2817-4cd1-b486-4b881f9d2ac6 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:16.940853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:16.941759 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:16.942049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:16.942350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] PATH_INFO: `/v3/users/5be22decb9bf4b9d8cdad50ba1d4b21f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:16.943653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5be22decb9bf4b9d8cdad50ba1d4b21f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:16.945589 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:16.987689 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:16.997334 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5be22decb9bf4b9d8cdad50ba1d4b21f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.998106 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5be22decb9bf4b9d8cdad50ba1d4b21f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:16.998807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5be22decb9bf4b9d8cdad50ba1d4b21f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:17.005703 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5be22decb9bf4b9d8cdad50ba1d4b21f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:17.017537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5be22decb9bf4b9d8cdad50ba1d4b21f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:17.031750 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1000/2035] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:16 2022] DELETE /identity/v3/users/5be22decb9bf4b9d8cdad50ba1d4b21f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:17.733571 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:17.735400 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:17.751497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b627f-e410-4d9c-bd1b-8cdb8a1a7394 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:17.822455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:17.849891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:17.851633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:17.852973 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:17.853264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] PATH_INFO: `/v3/projects/4ff0ce01ffa4413f8282e2ae8e817b90` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:17.855981 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff0ce01ffa4413f8282e2ae8e817b90)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:17.857939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:17.873272 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:17.874753 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:17.893692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16118e32-bf20-43f1-af18-c5ee98c0d353 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:17.908407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed9ce3-caf0-48d8-b502-ee676f0b0e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:17.910137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed9ce3-caf0-48d8-b502-ee676f0b0e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:17.910457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed9ce3-caf0-48d8-b502-ee676f0b0e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:17.914264 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed9ce3-caf0-48d8-b502-ee676f0b0e0c admin admin] PATH_INFO: `/v3/users/4c8d180b9dd2491fb4e5d8d500547734` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:17.915640 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed9ce3-caf0-48d8-b502-ee676f0b0e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c8d180b9dd2491fb4e5d8d500547734)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:17.922819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed9ce3-caf0-48d8-b502-ee676f0b0e0c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:17.944533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff0ce01ffa4413f8282e2ae8e817b90', 'request_id': 'req-c2939aae-334a-4353-aa64-ad6170eafd69'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:17.952063 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff0ce01ffa4413f8282e2ae8e817b90', 'request_id': 'req-c2939aae-334a-4353-aa64-ad6170eafd69'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:17.961440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff0ce01ffa4413f8282e2ae8e817b90', 'request_id': 'req-c2939aae-334a-4353-aa64-ad6170eafd69'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:17.991863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] The token cache is being invalidate because project 4ff0ce01ffa4413f8282e2ae8e817b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:17.992180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:17.994752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c2939aae-334a-4353-aa64-ad6170eafd69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:18.003934 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1036/2036] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:17 2022] DELETE /identity/v3/projects/4ff0ce01ffa4413f8282e2ae8e817b90 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:18.006118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da433b-a9d5-4570-9b50-79808defed78 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.006765 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da433b-a9d5-4570-9b50-79808defed78 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.007056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da433b-a9d5-4570-9b50-79808defed78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.025302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2eed9ce3-caf0-48d8-b502-ee676f0b0e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c8d180b9dd2491fb4e5d8d500547734'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:18.026406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2eed9ce3-caf0-48d8-b502-ee676f0b0e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c8d180b9dd2491fb4e5d8d500547734'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:18.032260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2eed9ce3-caf0-48d8-b502-ee676f0b0e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c8d180b9dd2491fb4e5d8d500547734'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:18.036735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2eed9ce3-caf0-48d8-b502-ee676f0b0e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c8d180b9dd2491fb4e5d8d500547734'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:18.048151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-28da433b-a9d5-4570-9b50-79808defed78 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:18.049593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-28da433b-a9d5-4570-9b50-79808defed78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:18.052028 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2eed9ce3-caf0-48d8-b502-ee676f0b0e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c8d180b9dd2491fb4e5d8d500547734'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:18.077265 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1001/2037] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:17 2022] DELETE /identity/v3/users/4c8d180b9dd2491fb4e5d8d500547734 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:18.083463 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216454f-ec6e-4ee4-be0f-3436b20ff8e2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.084044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216454f-ec6e-4ee4-be0f-3436b20ff8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.085002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216454f-ec6e-4ee4-be0f-3436b20ff8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.110884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9216454f-ec6e-4ee4-be0f-3436b20ff8e2 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:18.112746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9216454f-ec6e-4ee4-be0f-3436b20ff8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:18.169482 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1037/2038] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:18.187312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287465dc-5950-44e0-a2fe-204488a3a4ec None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.188346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287465dc-5950-44e0-a2fe-204488a3a4ec None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.188558 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287465dc-5950-44e0-a2fe-204488a3a4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.223007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-287465dc-5950-44e0-a2fe-204488a3a4ec None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:18.231855 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-287465dc-5950-44e0-a2fe-204488a3a4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:18.266341 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1002/2039] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:18.276787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216454f-ec6e-4ee4-be0f-3436b20ff8e2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:18.278563 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9216454f-ec6e-4ee4-be0f-3436b20ff8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:18.371162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:18.389639 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1038/2040] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:18.394314 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287465dc-5950-44e0-a2fe-204488a3a4ec None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:18.404772 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287465dc-5950-44e0-a2fe-204488a3a4ec None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:18.411854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bbedd-1996-49cc-9082-3c36279470cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:18.412837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bbedd-1996-49cc-9082-3c36279470cd admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.413233 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bbedd-1996-49cc-9082-3c36279470cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.413651 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bbedd-1996-49cc-9082-3c36279470cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.414798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bbedd-1996-49cc-9082-3c36279470cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:18.415262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bbedd-1996-49cc-9082-3c36279470cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:18.417335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bbedd-1996-49cc-9082-3c36279470cd admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:18.441176 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1039/2041] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:18.442361 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bbedd-1996-49cc-9082-3c36279470cd admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:18.447335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216454f-ec6e-4ee4-be0f-3436b20ff8e2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:18.451304 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b7959-8c6b-4211-bfce-ed562e7a6a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:18.452317 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b7959-8c6b-4211-bfce-ed562e7a6a33 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.452596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b7959-8c6b-4211-bfce-ed562e7a6a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.452863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b7959-8c6b-4211-bfce-ed562e7a6a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.465413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b7959-8c6b-4211-bfce-ed562e7a6a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:18.467780 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b7959-8c6b-4211-bfce-ed562e7a6a33 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:18.473826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bbedd-1996-49cc-9082-3c36279470cd admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:18.484600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60662f02-053f-4dbf-b512-b916a7c248ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:18.485291 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662f02-053f-4dbf-b512-b916a7c248ea admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.485613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662f02-053f-4dbf-b512-b916a7c248ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.485888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662f02-053f-4dbf-b512-b916a7c248ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.486615 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60662f02-053f-4dbf-b512-b916a7c248ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:18.489185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60662f02-053f-4dbf-b512-b916a7c248ea admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:18.495030 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1003/2042] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:18.515225 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:18.543946 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1040/2043] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:18 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:18.545215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b7959-8c6b-4211-bfce-ed562e7a6a33 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:18.560071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b7959-8c6b-4211-bfce-ed562e7a6a33 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:18.662467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60662f02-053f-4dbf-b512-b916a7c248ea admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:18.664553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-60662f02-053f-4dbf-b512-b916a7c248ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:18.672718 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49423ce-d01c-4297-9f50-e6b1788c074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:18.674290 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49423ce-d01c-4297-9f50-e6b1788c074c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.674585 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49423ce-d01c-4297-9f50-e6b1788c074c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.674866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49423ce-d01c-4297-9f50-e6b1788c074c admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.677394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60662f02-053f-4dbf-b512-b916a7c248ea admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:18.678614 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49423ce-d01c-4297-9f50-e6b1788c074c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:18.680585 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49423ce-d01c-4297-9f50-e6b1788c074c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:18.685047 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b3495-5ab1-40df-a22f-4d396482f2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:18.685047 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b3495-5ab1-40df-a22f-4d396482f2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.685047 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b3495-5ab1-40df-a22f-4d396482f2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.685047 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b3495-5ab1-40df-a22f-4d396482f2a7 admin admin] PATH_INFO: `/v3/users/5311020be0ee491c810919e3665b613e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.688016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b3495-5ab1-40df-a22f-4d396482f2a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5311020be0ee491c810919e3665b613e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:18.689954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b3495-5ab1-40df-a22f-4d396482f2a7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:18.735444 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1004/2044] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:18 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:18.736601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49423ce-d01c-4297-9f50-e6b1788c074c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:18.747562 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49423ce-d01c-4297-9f50-e6b1788c074c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:18.752497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89d543-10d0-4aee-803e-6be523c2bc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:18.753684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89d543-10d0-4aee-803e-6be523c2bc79 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.753971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89d543-10d0-4aee-803e-6be523c2bc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.754272 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89d543-10d0-4aee-803e-6be523c2bc79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.755029 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89d543-10d0-4aee-803e-6be523c2bc79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:18.756809 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89d543-10d0-4aee-803e-6be523c2bc79 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:18.757283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-760b3495-5ab1-40df-a22f-4d396482f2a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5311020be0ee491c810919e3665b613e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:18.758757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-760b3495-5ab1-40df-a22f-4d396482f2a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5311020be0ee491c810919e3665b613e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:18.770819 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1005/2045] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:18.775596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89d543-10d0-4aee-803e-6be523c2bc79 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:18.777413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5b89d543-10d0-4aee-803e-6be523c2bc79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:18.781059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-760b3495-5ab1-40df-a22f-4d396482f2a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5311020be0ee491c810919e3665b613e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:18.790356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-760b3495-5ab1-40df-a22f-4d396482f2a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5311020be0ee491c810919e3665b613e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:18.790593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89d543-10d0-4aee-803e-6be523c2bc79 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:18.791129 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-760b3495-5ab1-40df-a22f-4d396482f2a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5311020be0ee491c810919e3665b613e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:18.795906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350bdd0-59d2-4374-b3c5-81d16f830482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:18.796182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350bdd0-59d2-4374-b3c5-81d16f830482 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.796456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350bdd0-59d2-4374-b3c5-81d16f830482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.796870 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350bdd0-59d2-4374-b3c5-81d16f830482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.797518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350bdd0-59d2-4374-b3c5-81d16f830482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:18.799397 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350bdd0-59d2-4374-b3c5-81d16f830482 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:18.807421 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1041/2046] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:18 2022] DELETE /identity/v3/users/5311020be0ee491c810919e3665b613e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:18.811871 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6cbb4-1aed-4c7d-b9d3-0cc08ade0786 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.812168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6cbb4-1aed-4c7d-b9d3-0cc08ade0786 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.812447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6cbb4-1aed-4c7d-b9d3-0cc08ade0786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.813479 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1006/2047] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:18.817925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350bdd0-59d2-4374-b3c5-81d16f830482 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:18.828141 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350bdd0-59d2-4374-b3c5-81d16f830482 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:18.840148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-98c6cbb4-1aed-4c7d-b9d3-0cc08ade0786 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:18.841750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-98c6cbb4-1aed-4c7d-b9d3-0cc08ade0786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:18.894950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194da95-819c-471d-9a8f-d7776ab8642e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:18.895860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194da95-819c-471d-9a8f-d7776ab8642e admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.896135 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194da95-819c-471d-9a8f-d7776ab8642e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.896416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194da95-819c-471d-9a8f-d7776ab8642e admin admin] PATH_INFO: `/v3/projects/c69709409a0c4bc799732ea0416e6133/users/2e691f2a13344c84bbe368a0c8239ab3/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.903811 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1042/2048] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:18.907060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6cbb4-1aed-4c7d-b9d3-0cc08ade0786 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:18.913482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-98c6cbb4-1aed-4c7d-b9d3-0cc08ade0786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:18.917049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194da95-819c-471d-9a8f-d7776ab8642e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69709409a0c4bc799732ea0416e6133, user_id=2e691f2a13344c84bbe368a0c8239ab3, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:18.919812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194da95-819c-471d-9a8f-d7776ab8642e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:18.927212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6cbb4-1aed-4c7d-b9d3-0cc08ade0786 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:18.932329 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:18.933380 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:18.933795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:18.934076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] PATH_INFO: `/v3/projects/3c105b455fba44b19c7d56f8e817130f` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:18.935307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c105b455fba44b19c7d56f8e817130f)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:18.937268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:18.942026 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1007/2049] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:18 2022] PUT /identity/v3/projects/c69709409a0c4bc799732ea0416e6133/users/2e691f2a13344c84bbe368a0c8239ab3/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:18.942362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194da95-819c-471d-9a8f-d7776ab8642e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:18.946120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a194da95-819c-471d-9a8f-d7776ab8642e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:18.981473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:18.993366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c105b455fba44b19c7d56f8e817130f', 'request_id': 'req-21d42230-446d-4752-81fb-9febc0d5f6e3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.003826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c105b455fba44b19c7d56f8e817130f', 'request_id': 'req-21d42230-446d-4752-81fb-9febc0d5f6e3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.006969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:INSERT=2,keystone:SELECT=82,keystone:DELETE=16 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:19.011964 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c105b455fba44b19c7d56f8e817130f', 'request_id': 'req-21d42230-446d-4752-81fb-9febc0d5f6e3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.019707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194da95-819c-471d-9a8f-d7776ab8642e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:19.033728 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] The token cache is being invalidate because project 3c105b455fba44b19c7d56f8e817130f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:19.034764 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.035577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-21d42230-446d-4752-81fb-9febc0d5f6e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.037871 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1043/2050] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:18 2022] DELETE /identity/v3/projects/3c105b455fba44b19c7d56f8e817130f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:19.042755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a968fb-d759-4f32-abec-e060dde14227 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:19.043048 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a968fb-d759-4f32-abec-e060dde14227 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:19.043330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a968fb-d759-4f32-abec-e060dde14227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:19.080347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-50a968fb-d759-4f32-abec-e060dde14227 None None] MFA Rules not processed for user `2e691f2a13344c84bbe368a0c8239ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:19.081788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-50a968fb-d759-4f32-abec-e060dde14227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:19.101812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b65104-6fe8-4b1e-8cb2-44124e00f660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:19.102929 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b65104-6fe8-4b1e-8cb2-44124e00f660 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:19.103291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b65104-6fe8-4b1e-8cb2-44124e00f660 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:19.103746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b65104-6fe8-4b1e-8cb2-44124e00f660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:19.105948 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-46b65104-6fe8-4b1e-8cb2-44124e00f660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:19.171985 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1044/2051] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:19.175741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a968fb-d759-4f32-abec-e060dde14227 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:19.175741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-50a968fb-d759-4f32-abec-e060dde14227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:19.189605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b65104-6fe8-4b1e-8cb2-44124e00f660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:19.191347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b65104-6fe8-4b1e-8cb2-44124e00f660 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:19.202819 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1008/2052] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:19.203471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b65104-6fe8-4b1e-8cb2-44124e00f660 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:19.204933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-46b65104-6fe8-4b1e-8cb2-44124e00f660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:19.219624 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b65104-6fe8-4b1e-8cb2-44124e00f660 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:19.225755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138e164-e5d0-41af-9061-dcb6e68ec226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:19.226475 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138e164-e5d0-41af-9061-dcb6e68ec226 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:19.226798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138e164-e5d0-41af-9061-dcb6e68ec226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:19.227029 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138e164-e5d0-41af-9061-dcb6e68ec226 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:19.227742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138e164-e5d0-41af-9061-dcb6e68ec226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:19.231325 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138e164-e5d0-41af-9061-dcb6e68ec226 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:19.266375 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1009/2053] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:19 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:19.267466 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138e164-e5d0-41af-9061-dcb6e68ec226 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:19.272397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a968fb-d759-4f32-abec-e060dde14227 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:19.339365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:19.340919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:19.341207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:19.341572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] PATH_INFO: `/v3/users/0dcceded8b0f4a1bb4af81ee7ada8b47` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:19.343562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dcceded8b0f4a1bb4af81ee7ada8b47)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:19.344149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138e164-e5d0-41af-9061-dcb6e68ec226 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:19.345498 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:19.349167 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0117d2-c4a7-4817-8bb5-8636ee41c3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:19.350143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0117d2-c4a7-4817-8bb5-8636ee41c3d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:19.350517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0117d2-c4a7-4817-8bb5-8636ee41c3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:19.350897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0117d2-c4a7-4817-8bb5-8636ee41c3d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:19.351737 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0117d2-c4a7-4817-8bb5-8636ee41c3d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:19.353914 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0117d2-c4a7-4817-8bb5-8636ee41c3d8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:19.364324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:19.381219 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1010/2054] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:19 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:19.385191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0117d2-c4a7-4817-8bb5-8636ee41c3d8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:19.396190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0117d2-c4a7-4817-8bb5-8636ee41c3d8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:19.399866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:19.401437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23753b67-22f0-46a3-960b-acae0f3857dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:19.402312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23753b67-22f0-46a3-960b-acae0f3857dd admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:19.402596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23753b67-22f0-46a3-960b-acae0f3857dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:19.402863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23753b67-22f0-46a3-960b-acae0f3857dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:19.403715 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23753b67-22f0-46a3-960b-acae0f3857dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:19.405549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23753b67-22f0-46a3-960b-acae0f3857dd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:19.417368 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1011/2055] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:19.421604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23753b67-22f0-46a3-960b-acae0f3857dd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:19.425832 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dcceded8b0f4a1bb4af81ee7ada8b47'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.427158 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dcceded8b0f4a1bb4af81ee7ada8b47'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.432287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23753b67-22f0-46a3-960b-acae0f3857dd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:19.437772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ca8a-047b-4839-85ee-a5d832a604ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:19.438877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ca8a-047b-4839-85ee-a5d832a604ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:19.439164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ca8a-047b-4839-85ee-a5d832a604ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:19.439443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ca8a-047b-4839-85ee-a5d832a604ee admin admin] PATH_INFO: `/v3/projects/e516117df3ac4fa4adb9aa817f9e0f37/users/60b9c1d82656487884e176c089184009/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:19.445351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dcceded8b0f4a1bb4af81ee7ada8b47'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.454325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dcceded8b0f4a1bb4af81ee7ada8b47'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.455631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dcceded8b0f4a1bb4af81ee7ada8b47'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.456712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2ca8a-047b-4839-85ee-a5d832a604ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e516117df3ac4fa4adb9aa817f9e0f37, user_id=60b9c1d82656487884e176c089184009, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:19.462461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2ca8a-047b-4839-85ee-a5d832a604ee admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:19.477797 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1012/2056] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:19 2022] PUT /identity/v3/projects/e516117df3ac4fa4adb9aa817f9e0f37/users/60b9c1d82656487884e176c089184009/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:19.481540 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ca8a-047b-4839-85ee-a5d832a604ee admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:19.481679 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1045/2057] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:19 2022] DELETE /identity/v3/users/0dcceded8b0f4a1bb4af81ee7ada8b47 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:19.483648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-91a2ca8a-047b-4839-85ee-a5d832a604ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:19.521800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:19.523374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:19.569056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ca8a-047b-4839-85ee-a5d832a604ee admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:19.597182 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4ef50-2207-40be-8695-fe6443e79b31 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:19.603285 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:19.604086 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:19.604479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:19.604927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] PATH_INFO: `/v3/projects/7ee670637eec4eda9a7b6462cc5b5069` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:19.606078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee670637eec4eda9a7b6462cc5b5069)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:19.608136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:19.664174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e28f2-a9c3-4091-9f58-3000f2486724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:19.664878 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e28f2-a9c3-4091-9f58-3000f2486724 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:19.665185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e28f2-a9c3-4091-9f58-3000f2486724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:19.665432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e28f2-a9c3-4091-9f58-3000f2486724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:19.666796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e28f2-a9c3-4091-9f58-3000f2486724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:19.668768 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e28f2-a9c3-4091-9f58-3000f2486724 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:19.690115 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1013/2058] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:19.694840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e28f2-a9c3-4091-9f58-3000f2486724 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:19.715159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e28f2-a9c3-4091-9f58-3000f2486724 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:19.718358 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee670637eec4eda9a7b6462cc5b5069', 'request_id': 'req-6c41aeab-7caa-4122-ae28-e245e6157eb3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.731969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee670637eec4eda9a7b6462cc5b5069', 'request_id': 'req-6c41aeab-7caa-4122-ae28-e245e6157eb3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.743024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee670637eec4eda9a7b6462cc5b5069', 'request_id': 'req-6c41aeab-7caa-4122-ae28-e245e6157eb3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.772834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] The token cache is being invalidate because project 7ee670637eec4eda9a7b6462cc5b5069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:19.773367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.774389 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:19.776677 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1046/2059] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:19 2022] DELETE /identity/v3/projects/7ee670637eec4eda9a7b6462cc5b5069 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:19.781099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:19.783958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:19.808318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af21fe-b8cb-4dff-a9ac-9383db6cf6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:19.810064 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af21fe-b8cb-4dff-a9ac-9383db6cf6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:19.810477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af21fe-b8cb-4dff-a9ac-9383db6cf6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:19.810856 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af21fe-b8cb-4dff-a9ac-9383db6cf6dc admin admin] PATH_INFO: `/v3/projects/e516117df3ac4fa4adb9aa817f9e0f37/users/60b9c1d82656487884e176c089184009/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:19.814342 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af21fe-b8cb-4dff-a9ac-9383db6cf6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e516117df3ac4fa4adb9aa817f9e0f37, user_id=60b9c1d82656487884e176c089184009, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:19.817996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af21fe-b8cb-4dff-a9ac-9383db6cf6dc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:19.822126 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:19.840633 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1014/2060] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:19 2022] PUT /identity/v3/projects/e516117df3ac4fa4adb9aa817f9e0f37/users/60b9c1d82656487884e176c089184009/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:19.851423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced5f35-6971-4c52-9349-e649c551e336 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:19.851847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced5f35-6971-4c52-9349-e649c551e336 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:19.852397 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced5f35-6971-4c52-9349-e649c551e336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:19.870819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:19.900233 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bced5f35-6971-4c52-9349-e649c551e336 None None] MFA Rules not processed for user `60b9c1d82656487884e176c089184009`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:19.901706 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bced5f35-6971-4c52-9349-e649c551e336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:19.918528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41aeab-7caa-4122-ae28-e245e6157eb3 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:20.018306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:20.019427 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.019735 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.021060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] PATH_INFO: `/v3/users/0668c19227d847639574ab533a75e143` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.022616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=0668c19227d847639574ab533a75e143)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:20.024846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:20.026197 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1015/2061] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:20.038373 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced5f35-6971-4c52-9349-e649c551e336 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:20.040060 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bced5f35-6971-4c52-9349-e649c551e336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:20.058005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced5f35-6971-4c52-9349-e649c551e336 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:20.063407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:20.064813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.064813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.064813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] PATH_INFO: `/v3/projects/746e801af26d48ac943346f641ce9d90` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.066250 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=746e801af26d48ac943346f641ce9d90)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:20.068637 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:20.095543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0668c19227d847639574ab533a75e143'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.096235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0668c19227d847639574ab533a75e143'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.110597 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0668c19227d847639574ab533a75e143'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.118977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0668c19227d847639574ab533a75e143'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.119803 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0668c19227d847639574ab533a75e143'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.132537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '746e801af26d48ac943346f641ce9d90', 'request_id': 'req-10dca9de-891d-43ba-abe8-18d2d4431fbd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.135914 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1047/2062] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:19 2022] DELETE /identity/v3/users/0668c19227d847639574ab533a75e143 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:20.136914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:20.139162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:20.155488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '746e801af26d48ac943346f641ce9d90', 'request_id': 'req-10dca9de-891d-43ba-abe8-18d2d4431fbd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.162508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '746e801af26d48ac943346f641ce9d90', 'request_id': 'req-10dca9de-891d-43ba-abe8-18d2d4431fbd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.176883 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] The token cache is being invalidate because project 746e801af26d48ac943346f641ce9d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:20.177442 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.178232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-10dca9de-891d-43ba-abe8-18d2d4431fbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.180171 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1016/2063] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:20 2022] DELETE /identity/v3/projects/746e801af26d48ac943346f641ce9d90 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:20.188113 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cc0cf-a859-4191-a92c-283f24bf1728 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.188635 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cc0cf-a859-4191-a92c-283f24bf1728 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.188985 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cc0cf-a859-4191-a92c-283f24bf1728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.218353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6745e-2975-4535-911a-c3a2a86cc1ce admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:20.222661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df36440-6176-452e-a5b9-69ba9311f5d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:20.223334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36440-6176-452e-a5b9-69ba9311f5d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.223626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36440-6176-452e-a5b9-69ba9311f5d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.223729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8b6cc0cf-a859-4191-a92c-283f24bf1728 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:20.223904 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36440-6176-452e-a5b9-69ba9311f5d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.225600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8b6cc0cf-a859-4191-a92c-283f24bf1728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:20.226081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8df36440-6176-452e-a5b9-69ba9311f5d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:20.247262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:20.321116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df36440-6176-452e-a5b9-69ba9311f5d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:20.323208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df36440-6176-452e-a5b9-69ba9311f5d3 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:20.353693 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1048/2064] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:20.394283 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1017/2065] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:20.400534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1aa73-0e48-4583-b5c8-648228be02d7 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.400888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1aa73-0e48-4583-b5c8-648228be02d7 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.401223 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1aa73-0e48-4583-b5c8-648228be02d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.425372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-84e1aa73-0e48-4583-b5c8-648228be02d7 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:20.427953 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-84e1aa73-0e48-4583-b5c8-648228be02d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:20.441566 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1049/2066] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:20.446183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cc0cf-a859-4191-a92c-283f24bf1728 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:20.454542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:20.460180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cc0cf-a859-4191-a92c-283f24bf1728 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:20.497053 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c2f82-e265-4ff1-9f94-17153f7e1372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:20.497820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c2f82-e265-4ff1-9f94-17153f7e1372 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.498164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c2f82-e265-4ff1-9f94-17153f7e1372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.498497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c2f82-e265-4ff1-9f94-17153f7e1372 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.500746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c2f82-e265-4ff1-9f94-17153f7e1372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:20.501213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c2f82-e265-4ff1-9f94-17153f7e1372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:20.503087 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c2f82-e265-4ff1-9f94-17153f7e1372 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:20.522346 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1018/2067] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:20.527183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1aa73-0e48-4583-b5c8-648228be02d7 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:20.549734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1aa73-0e48-4583-b5c8-648228be02d7 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:20.559449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec6036-97cc-4843-a2a7-643bdd89a363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:20.560474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec6036-97cc-4843-a2a7-643bdd89a363 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.560474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec6036-97cc-4843-a2a7-643bdd89a363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.560697 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec6036-97cc-4843-a2a7-643bdd89a363 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.561542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec6036-97cc-4843-a2a7-643bdd89a363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:20.566592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec6036-97cc-4843-a2a7-643bdd89a363 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:20.620074 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1050/2068] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:20 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:20.624757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c2f82-e265-4ff1-9f94-17153f7e1372 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:20.633444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec6036-97cc-4843-a2a7-643bdd89a363 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:20.635552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f1ec6036-97cc-4843-a2a7-643bdd89a363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:20.654679 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c2f82-e265-4ff1-9f94-17153f7e1372 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:20.715587 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec6036-97cc-4843-a2a7-643bdd89a363 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:20.720383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:20.720761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30194b2f-81af-4263-9301-59fdebcd0b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:20.721266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.721437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30194b2f-81af-4263-9301-59fdebcd0b65 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.721541 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.721711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30194b2f-81af-4263-9301-59fdebcd0b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.721849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] PATH_INFO: `/v3/projects/a17db58263f844af825f4d4c043b4ad0` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.721970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30194b2f-81af-4263-9301-59fdebcd0b65 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.722713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30194b2f-81af-4263-9301-59fdebcd0b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:20.723074 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a17db58263f844af825f4d4c043b4ad0)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:20.724627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30194b2f-81af-4263-9301-59fdebcd0b65 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:20.725001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:20.745087 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1019/2069] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:20 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:20.746173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30194b2f-81af-4263-9301-59fdebcd0b65 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:20.747569 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-30194b2f-81af-4263-9301-59fdebcd0b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:20.798408 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a17db58263f844af825f4d4c043b4ad0', 'request_id': 'req-597d512d-7f4b-4863-aa3f-78df0f014aba'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.804184 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a17db58263f844af825f4d4c043b4ad0', 'request_id': 'req-597d512d-7f4b-4863-aa3f-78df0f014aba'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.810501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a17db58263f844af825f4d4c043b4ad0', 'request_id': 'req-597d512d-7f4b-4863-aa3f-78df0f014aba'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.811149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30194b2f-81af-4263-9301-59fdebcd0b65 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:20.823449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] The token cache is being invalidate because project a17db58263f844af825f4d4c043b4ad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:20.823737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.824355 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:20.826161 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1051/2070] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:20 2022] DELETE /identity/v3/projects/a17db58263f844af825f4d4c043b4ad0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:20.826772 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:20.828133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:20.842971 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:20.864484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce209a-0d1d-4814-bdd1-140496b87049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:20.865271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce209a-0d1d-4814-bdd1-140496b87049 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.865623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce209a-0d1d-4814-bdd1-140496b87049 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.865906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce209a-0d1d-4814-bdd1-140496b87049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.867879 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4cce209a-0d1d-4814-bdd1-140496b87049 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:20.895330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d512d-7f4b-4863-aa3f-78df0f014aba admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:20.899961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223a2d3-f974-4960-b6fe-1f66d761313e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:20.900650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223a2d3-f974-4960-b6fe-1f66d761313e admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.900952 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223a2d3-f974-4960-b6fe-1f66d761313e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.901254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223a2d3-f974-4960-b6fe-1f66d761313e admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.901975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223a2d3-f974-4960-b6fe-1f66d761313e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:20.903642 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223a2d3-f974-4960-b6fe-1f66d761313e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:20.914058 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1052/2071] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:20.915379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304e530-2927-4950-a60f-3a6eaceea987 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.915666 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304e530-2927-4950-a60f-3a6eaceea987 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.915937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304e530-2927-4950-a60f-3a6eaceea987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.919979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:20.934281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce209a-0d1d-4814-bdd1-140496b87049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:20.935953 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce209a-0d1d-4814-bdd1-140496b87049 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:20.941353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-8304e530-2927-4950-a60f-3a6eaceea987 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:20.942774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8304e530-2927-4950-a60f-3a6eaceea987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:20.946534 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1020/2072] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:20.947294 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce209a-0d1d-4814-bdd1-140496b87049 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:20.953500 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1053/2073] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:20.958006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce209a-0d1d-4814-bdd1-140496b87049 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:20.961249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42284555-c292-4549-8246-73a171bbfc66 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.961721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42284555-c292-4549-8246-73a171bbfc66 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.962078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42284555-c292-4549-8246-73a171bbfc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.963618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a98ad-cc3b-465a-8499-cbe83c77cbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:20.964317 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a98ad-cc3b-465a-8499-cbe83c77cbde admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:20.964602 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a98ad-cc3b-465a-8499-cbe83c77cbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:20.964884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a98ad-cc3b-465a-8499-cbe83c77cbde admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:20.965632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a98ad-cc3b-465a-8499-cbe83c77cbde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:20.967282 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a98ad-cc3b-465a-8499-cbe83c77cbde admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:20.981046 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-42284555-c292-4549-8246-73a171bbfc66 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:20.982267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-42284555-c292-4549-8246-73a171bbfc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:20.987222 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1021/2074] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:20.988496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a98ad-cc3b-465a-8499-cbe83c77cbde admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:20.993382 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1054/2075] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:20.997259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42284555-c292-4549-8246-73a171bbfc66 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:21.005256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a98ad-cc3b-465a-8499-cbe83c77cbde admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.007797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42284555-c292-4549-8246-73a171bbfc66 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.013123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd026010-caae-445b-8914-908499724dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.013524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a7c14-e579-4c00-937e-29657b8b5556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.013797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026010-caae-445b-8914-908499724dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.014192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026010-caae-445b-8914-908499724dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.014540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026010-caae-445b-8914-908499724dfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.014701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a7c14-e579-4c00-937e-29657b8b5556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.015442 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a7c14-e579-4c00-937e-29657b8b5556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.015898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a7c14-e579-4c00-937e-29657b8b5556 admin admin] PATH_INFO: `/v3/projects/1ad13e7306d74bb68b895ec28c1bfd3c/users/bb41d8ee37f8426d9ac14efbbda441a5/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.031022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd026010-caae-445b-8914-908499724dfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:21.031541 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd026010-caae-445b-8914-908499724dfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.038005 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd026010-caae-445b-8914-908499724dfe admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.042100 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a7c14-e579-4c00-937e-29657b8b5556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad13e7306d74bb68b895ec28c1bfd3c, user_id=bb41d8ee37f8426d9ac14efbbda441a5, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.045512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a7c14-e579-4c00-937e-29657b8b5556 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.053262 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1055/2076] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:13:21.058278 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd026010-caae-445b-8914-908499724dfe admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:21.065638 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1022/2077] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:20 2022] PUT /identity/v3/projects/1ad13e7306d74bb68b895ec28c1bfd3c/users/bb41d8ee37f8426d9ac14efbbda441a5/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:21.072174 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd026010-caae-445b-8914-908499724dfe admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.076593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5e0e7-866d-4cc4-aa2a-a481d4ce8976 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.076887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5e0e7-866d-4cc4-aa2a-a481d4ce8976 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.077178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5e0e7-866d-4cc4-aa2a-a481d4ce8976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.121590 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-01c5e0e7-866d-4cc4-aa2a-a481d4ce8976 None None] MFA Rules not processed for user `bb41d8ee37f8426d9ac14efbbda441a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:21.123285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-01c5e0e7-866d-4cc4-aa2a-a481d4ce8976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:21.164345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238cdb6-8a3f-4f73-b4e8-add8d9ef4a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.165060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238cdb6-8a3f-4f73-b4e8-add8d9ef4a40 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.165345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238cdb6-8a3f-4f73-b4e8-add8d9ef4a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.165623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238cdb6-8a3f-4f73-b4e8-add8d9ef4a40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.166400 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238cdb6-8a3f-4f73-b4e8-add8d9ef4a40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.171675 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238cdb6-8a3f-4f73-b4e8-add8d9ef4a40 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.219021 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1023/2078] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:21.226130 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1056/2079] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:21 2022] POST /identity/v3/projects => generated 389 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:21.230910 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5e0e7-866d-4cc4-aa2a-a481d4ce8976 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:21.245404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238cdb6-8a3f-4f73-b4e8-add8d9ef4a40 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:21.246056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5e0e7-866d-4cc4-aa2a-a481d4ce8976 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.247392 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3238cdb6-8a3f-4f73-b4e8-add8d9ef4a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:21.310827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238cdb6-8a3f-4f73-b4e8-add8d9ef4a40 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.315687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39865ed6-1ea6-4978-8b23-b2731466c145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.316392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39865ed6-1ea6-4978-8b23-b2731466c145 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.316685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39865ed6-1ea6-4978-8b23-b2731466c145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.316971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39865ed6-1ea6-4978-8b23-b2731466c145 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.317375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d57a72-6c6a-48a8-869e-8fe6f9fef745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.317696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39865ed6-1ea6-4978-8b23-b2731466c145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.318718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d57a72-6c6a-48a8-869e-8fe6f9fef745 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.319009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d57a72-6c6a-48a8-869e-8fe6f9fef745 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.319667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39865ed6-1ea6-4978-8b23-b2731466c145 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.320019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d57a72-6c6a-48a8-869e-8fe6f9fef745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.343461 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1024/2080] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:21 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:21.344518 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:21.351931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39865ed6-1ea6-4978-8b23-b2731466c145 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:21.368306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39865ed6-1ea6-4978-8b23-b2731466c145 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.373846 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd198905-93bd-4c97-8eca-0fc68996ed02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.374641 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd198905-93bd-4c97-8eca-0fc68996ed02 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.374995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd198905-93bd-4c97-8eca-0fc68996ed02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.375363 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd198905-93bd-4c97-8eca-0fc68996ed02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.376173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd198905-93bd-4c97-8eca-0fc68996ed02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.378013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd198905-93bd-4c97-8eca-0fc68996ed02 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.392762 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1025/2081] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:21.393172 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd198905-93bd-4c97-8eca-0fc68996ed02 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:21.398657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d57a72-6c6a-48a8-869e-8fe6f9fef745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.400103 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d57a72-6c6a-48a8-869e-8fe6f9fef745 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.403935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd198905-93bd-4c97-8eca-0fc68996ed02 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.409240 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed90a7c-c7dd-41f7-ba5e-e3d29abd17e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.409984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed90a7c-c7dd-41f7-ba5e-e3d29abd17e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.410402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed90a7c-c7dd-41f7-ba5e-e3d29abd17e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.410747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed90a7c-c7dd-41f7-ba5e-e3d29abd17e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.411551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed90a7c-c7dd-41f7-ba5e-e3d29abd17e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.413220 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed90a7c-c7dd-41f7-ba5e-e3d29abd17e9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.416496 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1057/2082] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:21.425937 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1026/2083] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:21.430094 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d57a72-6c6a-48a8-869e-8fe6f9fef745 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:21.441929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d57a72-6c6a-48a8-869e-8fe6f9fef745 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.453313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864fb78-e0f9-4384-a7c1-fc493303cdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.454254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864fb78-e0f9-4384-a7c1-fc493303cdc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.454254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864fb78-e0f9-4384-a7c1-fc493303cdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.454437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864fb78-e0f9-4384-a7c1-fc493303cdc8 admin admin] PATH_INFO: `/v3/projects/16ee8fd269bd4879aeb6ca2a1910ed3a/users/b8553d1d36544a559166bf86be71cfb8/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.482397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864fb78-e0f9-4384-a7c1-fc493303cdc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16ee8fd269bd4879aeb6ca2a1910ed3a, user_id=b8553d1d36544a559166bf86be71cfb8, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.485172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864fb78-e0f9-4384-a7c1-fc493303cdc8 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.490471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed90a7c-c7dd-41f7-ba5e-e3d29abd17e9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:21.492877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3ed90a7c-c7dd-41f7-ba5e-e3d29abd17e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:21.514579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed90a7c-c7dd-41f7-ba5e-e3d29abd17e9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.527739 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1058/2084] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:21 2022] PUT /identity/v3/projects/16ee8fd269bd4879aeb6ca2a1910ed3a/users/b8553d1d36544a559166bf86be71cfb8/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:21.540190 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120744cc-533a-47ed-8ee6-f53255f71051 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.540527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120744cc-533a-47ed-8ee6-f53255f71051 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.541947 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120744cc-533a-47ed-8ee6-f53255f71051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.552107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:21.600154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-120744cc-533a-47ed-8ee6-f53255f71051 None None] MFA Rules not processed for user `b8553d1d36544a559166bf86be71cfb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:21.602006 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-120744cc-533a-47ed-8ee6-f53255f71051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:21.615961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e88184-720c-4228-9cda-888839aa87e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.616978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e88184-720c-4228-9cda-888839aa87e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.617305 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e88184-720c-4228-9cda-888839aa87e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.617522 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e88184-720c-4228-9cda-888839aa87e3 admin admin] PATH_INFO: `/v3/users/2e691f2a13344c84bbe368a0c8239ab3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.618902 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e88184-720c-4228-9cda-888839aa87e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e691f2a13344c84bbe368a0c8239ab3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.622117 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e88184-720c-4228-9cda-888839aa87e3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.690350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d6e88184-720c-4228-9cda-888839aa87e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e691f2a13344c84bbe368a0c8239ab3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:21.691130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d6e88184-720c-4228-9cda-888839aa87e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e691f2a13344c84bbe368a0c8239ab3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:21.692003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d6e88184-720c-4228-9cda-888839aa87e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e691f2a13344c84bbe368a0c8239ab3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:21.706526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d6e88184-720c-4228-9cda-888839aa87e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e691f2a13344c84bbe368a0c8239ab3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:21.709933 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1059/2085] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:21.714760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120744cc-533a-47ed-8ee6-f53255f71051 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:21.723830 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d6e88184-720c-4228-9cda-888839aa87e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e691f2a13344c84bbe368a0c8239ab3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:21.728242 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120744cc-533a-47ed-8ee6-f53255f71051 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.736168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82709b7-1e65-4388-ad87-595aab599348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.736950 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82709b7-1e65-4388-ad87-595aab599348 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.737298 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82709b7-1e65-4388-ad87-595aab599348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.737582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82709b7-1e65-4388-ad87-595aab599348 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.738434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82709b7-1e65-4388-ad87-595aab599348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.741505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82709b7-1e65-4388-ad87-595aab599348 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.747665 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1027/2086] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:21 2022] DELETE /identity/v3/users/2e691f2a13344c84bbe368a0c8239ab3 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:21.752776 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec4b9e-0cae-4563-8fc7-9c9368feba36 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.754748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec4b9e-0cae-4563-8fc7-9c9368feba36 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.755282 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec4b9e-0cae-4563-8fc7-9c9368feba36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.783406 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1060/2087] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:21 2022] POST /identity/v3/projects => generated 389 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:21.788093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82709b7-1e65-4388-ad87-595aab599348 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:21.789954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f82709b7-1e65-4388-ad87-595aab599348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:21.812566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9cec4b9e-0cae-4563-8fc7-9c9368feba36 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:21.813915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9cec4b9e-0cae-4563-8fc7-9c9368feba36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:21.934920 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1028/2088] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:21.936288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec4b9e-0cae-4563-8fc7-9c9368feba36 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:21.945345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9cec4b9e-0cae-4563-8fc7-9c9368feba36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:21.959318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec4b9e-0cae-4563-8fc7-9c9368feba36 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.962426 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82709b7-1e65-4388-ad87-595aab599348 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:21.964662 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.965550 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.965834 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.966105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] PATH_INFO: `/v3/users/fb18afcd78c34b4d84b00f79020664c7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.967329 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb18afcd78c34b4d84b00f79020664c7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.969133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16455f-f577-43b6-a91e-adf190d21ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:21.969278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.970132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16455f-f577-43b6-a91e-adf190d21ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:21.970595 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16455f-f577-43b6-a91e-adf190d21ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:21.971114 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16455f-f577-43b6-a91e-adf190d21ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:21.972145 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16455f-f577-43b6-a91e-adf190d21ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:21.974420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16455f-f577-43b6-a91e-adf190d21ef4 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:21.986178 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:22.006607 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1061/2089] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:21 2022] POST /identity/v3/users => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:22.010892 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:22.012605 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16455f-f577-43b6-a91e-adf190d21ef4 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:22.016338 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0d16455f-f577-43b6-a91e-adf190d21ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:22.042009 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb18afcd78c34b4d84b00f79020664c7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.042908 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb18afcd78c34b4d84b00f79020664c7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.043655 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb18afcd78c34b4d84b00f79020664c7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.050981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb18afcd78c34b4d84b00f79020664c7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.064120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb18afcd78c34b4d84b00f79020664c7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.080365 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1029/2090] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:21 2022] DELETE /identity/v3/users/fb18afcd78c34b4d84b00f79020664c7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:22.080924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:22.082306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:22.083808 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16455f-f577-43b6-a91e-adf190d21ef4 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:22.133535 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be3157-4ccd-421b-aa8a-fa61ffcd5d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:22.134252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be3157-4ccd-421b-aa8a-fa61ffcd5d7b service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.134550 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be3157-4ccd-421b-aa8a-fa61ffcd5d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.134830 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be3157-4ccd-421b-aa8a-fa61ffcd5d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.136795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-14be3157-4ccd-421b-aa8a-fa61ffcd5d7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:22.148155 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d9477-4fa4-40df-adbd-88aa1696aa10 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:22.148682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be3157-4ccd-421b-aa8a-fa61ffcd5d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:22.150237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be3157-4ccd-421b-aa8a-fa61ffcd5d7b service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:22.153099 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72af08-8cd2-4ff4-aacd-6deb3dce789d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:22.153770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72af08-8cd2-4ff4-aacd-6deb3dce789d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.154056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72af08-8cd2-4ff4-aacd-6deb3dce789d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.154382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72af08-8cd2-4ff4-aacd-6deb3dce789d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.155109 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72af08-8cd2-4ff4-aacd-6deb3dce789d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:22.156699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72af08-8cd2-4ff4-aacd-6deb3dce789d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:22.160063 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1062/2091] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:22.161322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126cb4c-8115-432a-891b-e7d6281c9c44 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.161671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126cb4c-8115-432a-891b-e7d6281c9c44 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.162321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126cb4c-8115-432a-891b-e7d6281c9c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.167300 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1030/2092] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:22.171028 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72af08-8cd2-4ff4-aacd-6deb3dce789d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:22.181396 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72af08-8cd2-4ff4-aacd-6deb3dce789d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:22.183215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b126cb4c-8115-432a-891b-e7d6281c9c44 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:22.184493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b126cb4c-8115-432a-891b-e7d6281c9c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:22.186711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183e536-2458-418e-9473-3ffd8d8ff9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:22.187617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183e536-2458-418e-9473-3ffd8d8ff9f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.187898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183e536-2458-418e-9473-3ffd8d8ff9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.188180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183e536-2458-418e-9473-3ffd8d8ff9f0 admin admin] PATH_INFO: `/v3/projects/fc71b981cab0426b9190331efea714c7/users/6bb3f5c4b1584b52871c7aa66241834b/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.193834 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1063/2093] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:22.204860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183e536-2458-418e-9473-3ffd8d8ff9f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc71b981cab0426b9190331efea714c7, user_id=6bb3f5c4b1584b52871c7aa66241834b, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:22.208058 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126cb4c-8115-432a-891b-e7d6281c9c44 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:22.210152 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b126cb4c-8115-432a-891b-e7d6281c9c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:22.215595 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183e536-2458-418e-9473-3ffd8d8ff9f0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:22.225044 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126cb4c-8115-432a-891b-e7d6281c9c44 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:22.232647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414bd75-e5cb-4c2a-8b81-f048af606b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:22.233829 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414bd75-e5cb-4c2a-8b81-f048af606b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.234203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414bd75-e5cb-4c2a-8b81-f048af606b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.234611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414bd75-e5cb-4c2a-8b81-f048af606b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.240717 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1031/2094] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:22 2022] PUT /identity/v3/projects/fc71b981cab0426b9190331efea714c7/users/6bb3f5c4b1584b52871c7aa66241834b/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:22.244993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183e536-2458-418e-9473-3ffd8d8ff9f0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:22.248724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414bd75-e5cb-4c2a-8b81-f048af606b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:22.250424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414bd75-e5cb-4c2a-8b81-f048af606b98 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:22.260969 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1064/2095] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:22.309108 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183e536-2458-418e-9473-3ffd8d8ff9f0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:22.315493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04badf-a19f-4fe1-a693-dd323374c591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:22.316061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04badf-a19f-4fe1-a693-dd323374c591 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.316372 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04badf-a19f-4fe1-a693-dd323374c591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.317069 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04badf-a19f-4fe1-a693-dd323374c591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.317491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04badf-a19f-4fe1-a693-dd323374c591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:22.319150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04badf-a19f-4fe1-a693-dd323374c591 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:22.334977 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1032/2096] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:22.337442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414bd75-e5cb-4c2a-8b81-f048af606b98 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:22.348897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414bd75-e5cb-4c2a-8b81-f048af606b98 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:22.354150 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0a66f-4548-4ef1-8f60-f215baf9bc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:22.355059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0a66f-4548-4ef1-8f60-f215baf9bc1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.355351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0a66f-4548-4ef1-8f60-f215baf9bc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.355634 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0a66f-4548-4ef1-8f60-f215baf9bc1d admin admin] PATH_INFO: `/v3/projects/fc71b981cab0426b9190331efea714c7/users/6bb3f5c4b1584b52871c7aa66241834b/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.358118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0a66f-4548-4ef1-8f60-f215baf9bc1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc71b981cab0426b9190331efea714c7, user_id=6bb3f5c4b1584b52871c7aa66241834b, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:22.361363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0a66f-4548-4ef1-8f60-f215baf9bc1d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:22.376548 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1065/2097] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:22 2022] PUT /identity/v3/projects/fc71b981cab0426b9190331efea714c7/users/6bb3f5c4b1584b52871c7aa66241834b/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:22.388555 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86912bdb-97d6-4083-b6db-48ec8e7fdf4e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.388871 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86912bdb-97d6-4083-b6db-48ec8e7fdf4e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.389159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86912bdb-97d6-4083-b6db-48ec8e7fdf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.419584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:22.422258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-86912bdb-97d6-4083-b6db-48ec8e7fdf4e None None] MFA Rules not processed for user `6bb3f5c4b1584b52871c7aa66241834b`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:22.424661 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-86912bdb-97d6-4083-b6db-48ec8e7fdf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:22.527837 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1033/2098] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:22.540010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0a66f-4548-4ef1-8f60-f215baf9bc1d admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:22.541948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-29e0a66f-4548-4ef1-8f60-f215baf9bc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:22.544191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86912bdb-97d6-4083-b6db-48ec8e7fdf4e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:22.545988 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-86912bdb-97d6-4083-b6db-48ec8e7fdf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:22.664415 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0a66f-4548-4ef1-8f60-f215baf9bc1d admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:22.678011 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb14e6-36b1-43e9-9c3b-c38d1f5786db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:22.681060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb14e6-36b1-43e9-9c3b-c38d1f5786db service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.681354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb14e6-36b1-43e9-9c3b-c38d1f5786db service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.681633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb14e6-36b1-43e9-9c3b-c38d1f5786db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.684240 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86912bdb-97d6-4083-b6db-48ec8e7fdf4e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:22.689153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:22.690030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.690380 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.690704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] PATH_INFO: `/v3/projects/c69709409a0c4bc799732ea0416e6133` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.691892 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c69709409a0c4bc799732ea0416e6133)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:22.692153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb14e6-36b1-43e9-9c3b-c38d1f5786db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:22.693690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb14e6-36b1-43e9-9c3b-c38d1f5786db service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:22.693801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:22.710458 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1066/2099] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:22.711322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb14e6-36b1-43e9-9c3b-c38d1f5786db service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:22.712700 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c4eb14e6-36b1-43e9-9c3b-c38d1f5786db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:22.727926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb14e6-36b1-43e9-9c3b-c38d1f5786db service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:22.733212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:22.734204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.740670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.741081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] PATH_INFO: `/v3/projects/df0575c2dee64315851eac5f73862fa6` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.742490 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0575c2dee64315851eac5f73862fa6)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:22.745292 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:22.784346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:22.812512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c69709409a0c4bc799732ea0416e6133', 'request_id': 'req-4ef38082-f178-48a5-b4e5-d2424bdce359'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.819201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0575c2dee64315851eac5f73862fa6', 'request_id': 'req-149f3592-91ff-4634-99c1-ab59180851ff'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.823648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c69709409a0c4bc799732ea0416e6133', 'request_id': 'req-4ef38082-f178-48a5-b4e5-d2424bdce359'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.827674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0575c2dee64315851eac5f73862fa6', 'request_id': 'req-149f3592-91ff-4634-99c1-ab59180851ff'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.838712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0575c2dee64315851eac5f73862fa6', 'request_id': 'req-149f3592-91ff-4634-99c1-ab59180851ff'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.838876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c69709409a0c4bc799732ea0416e6133', 'request_id': 'req-4ef38082-f178-48a5-b4e5-d2424bdce359'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.852226 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] The token cache is being invalidate because project c69709409a0c4bc799732ea0416e6133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:22.852630 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.853418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.855702 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] The token cache is being invalidate because project df0575c2dee64315851eac5f73862fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:22.858730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.858958 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1034/2100] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:22 2022] DELETE /identity/v3/projects/c69709409a0c4bc799732ea0416e6133 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:22.859037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:22.859752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:22.861364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:22.863393 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1067/2101] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:22 2022] DELETE /identity/v3/projects/df0575c2dee64315851eac5f73862fa6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:22.868799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:22.870560 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:22.901030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:22.948101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef38082-f178-48a5-b4e5-d2424bdce359 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:22.949290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f3592-91ff-4634-99c1-ab59180851ff admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:22.955893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b09c8-36ba-4161-8b4c-c220169ca14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:22.956974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b09c8-36ba-4161-8b4c-c220169ca14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.958353 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:22.960263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:22.960263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.960263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] PATH_INFO: `/v3/users/71e6664b05ee44d18790aef2b4e480f1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.960885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b09c8-36ba-4161-8b4c-c220169ca14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:22.961161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b09c8-36ba-4161-8b4c-c220169ca14d admin admin] PATH_INFO: `/v3/users/60b9c1d82656487884e176c089184009` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:22.962495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b09c8-36ba-4161-8b4c-c220169ca14d admin admin] RBAC: Authorizing `identity:delete_user(user_id=60b9c1d82656487884e176c089184009)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:22.964614 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b09c8-36ba-4161-8b4c-c220169ca14d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:22.969341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71e6664b05ee44d18790aef2b4e480f1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:22.971351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:23.047630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71e6664b05ee44d18790aef2b4e480f1'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.048420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71e6664b05ee44d18790aef2b4e480f1'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.053353 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-080b09c8-36ba-4161-8b4c-c220169ca14d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60b9c1d82656487884e176c089184009'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.054555 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-080b09c8-36ba-4161-8b4c-c220169ca14d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60b9c1d82656487884e176c089184009'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.055342 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-080b09c8-36ba-4161-8b4c-c220169ca14d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60b9c1d82656487884e176c089184009'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.063374 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-080b09c8-36ba-4161-8b4c-c220169ca14d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60b9c1d82656487884e176c089184009'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.072263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71e6664b05ee44d18790aef2b4e480f1'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.075271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-080b09c8-36ba-4161-8b4c-c220169ca14d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60b9c1d82656487884e176c089184009'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.079059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71e6664b05ee44d18790aef2b4e480f1'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.079793 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71e6664b05ee44d18790aef2b4e480f1'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.090671 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1035/2102] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:22 2022] DELETE /identity/v3/users/60b9c1d82656487884e176c089184009 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:23.096316 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1068/2103] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:22 2022] DELETE /identity/v3/users/71e6664b05ee44d18790aef2b4e480f1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:23.096316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:23.096316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:23.098531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273db1f8-b3f2-4cd4-b632-4ccba148aa51 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:23.098983 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273db1f8-b3f2-4cd4-b632-4ccba148aa51 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:23.099406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273db1f8-b3f2-4cd4-b632-4ccba148aa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:23.118838 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-273db1f8-b3f2-4cd4-b632-4ccba148aa51 None None] MFA Rules not processed for user `bb41d8ee37f8426d9ac14efbbda441a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:23.120513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-273db1f8-b3f2-4cd4-b632-4ccba148aa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:23.131076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:23.195199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739077e-662f-4584-9c45-05bea7062921 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:23.209681 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1036/2104] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:23.209994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:23.209994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:23.209994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:23.209994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] PATH_INFO: `/v3/users/b8553d1d36544a559166bf86be71cfb8` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:23.210509 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8553d1d36544a559166bf86be71cfb8)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:23.212783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:23.313027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8553d1d36544a559166bf86be71cfb8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.314059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8553d1d36544a559166bf86be71cfb8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.329464 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8553d1d36544a559166bf86be71cfb8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.338486 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8553d1d36544a559166bf86be71cfb8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.339760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8553d1d36544a559166bf86be71cfb8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.363918 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1069/2105] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:23 2022] DELETE /identity/v3/users/b8553d1d36544a559166bf86be71cfb8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:23.368692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f2207-eebe-465e-a4f9-423c6fa6d373 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:23.368987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f2207-eebe-465e-a4f9-423c6fa6d373 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:23.369266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f2207-eebe-465e-a4f9-423c6fa6d373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:23.397812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8f6f2207-eebe-465e-a4f9-423c6fa6d373 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:23.399535 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8f6f2207-eebe-465e-a4f9-423c6fa6d373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:23.470939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:23.500857 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1037/2106] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:23.706985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:23.710077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:23.734124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fe921-1eda-48bf-bbe1-d606e180e6be admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:23.742565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:23.744073 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:23.744452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:23.744901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] PATH_INFO: `/v3/projects/e516117df3ac4fa4adb9aa817f9e0f37` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:23.747179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e516117df3ac4fa4adb9aa817f9e0f37)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:23.750533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:23.759611 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f2207-eebe-465e-a4f9-423c6fa6d373 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:23.763084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8f6f2207-eebe-465e-a4f9-423c6fa6d373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:23.829606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e516117df3ac4fa4adb9aa817f9e0f37', 'request_id': 'req-17319c2d-5d26-424a-b9ac-a5de95d1c52c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.838297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e516117df3ac4fa4adb9aa817f9e0f37', 'request_id': 'req-17319c2d-5d26-424a-b9ac-a5de95d1c52c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.850959 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e516117df3ac4fa4adb9aa817f9e0f37', 'request_id': 'req-17319c2d-5d26-424a-b9ac-a5de95d1c52c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.870778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] The token cache is being invalidate because project e516117df3ac4fa4adb9aa817f9e0f37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:23.871087 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.871919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-17319c2d-5d26-424a-b9ac-a5de95d1c52c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:23.874098 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1070/2107] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:23 2022] DELETE /identity/v3/projects/e516117df3ac4fa4adb9aa817f9e0f37 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:23.878939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f2207-eebe-465e-a4f9-423c6fa6d373 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:23.879553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37955798-3a1d-42b4-a4d7-b71c9eab217c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:23.880010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37955798-3a1d-42b4-a4d7-b71c9eab217c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:23.880502 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37955798-3a1d-42b4-a4d7-b71c9eab217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:23.906000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-37955798-3a1d-42b4-a4d7-b71c9eab217c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:23.911151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-37955798-3a1d-42b4-a4d7-b71c9eab217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:23.970974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6d432-fd88-4b9f-9675-b899503426e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:23.970974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6d432-fd88-4b9f-9675-b899503426e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:23.971653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6d432-fd88-4b9f-9675-b899503426e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:23.971653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6d432-fd88-4b9f-9675-b899503426e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:23.974094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b3d6d432-fd88-4b9f-9675-b899503426e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:23.990966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:24.054252 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1071/2108] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:24.070548 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dd7e0-2b86-429a-81e5-dc1cca61e276 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.070845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dd7e0-2b86-429a-81e5-dc1cca61e276 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.071127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dd7e0-2b86-429a-81e5-dc1cca61e276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.095386 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-413dd7e0-2b86-429a-81e5-dc1cca61e276 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:24.105621 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6d432-fd88-4b9f-9675-b899503426e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:24.105925 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-413dd7e0-2b86-429a-81e5-dc1cca61e276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:24.106012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6d432-fd88-4b9f-9675-b899503426e4 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:24.112795 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1072/2109] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:24.116665 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1038/2110] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:24.117208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dd7e0-2b86-429a-81e5-dc1cca61e276 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:24.126729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dd7e0-2b86-429a-81e5-dc1cca61e276 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:24.133455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca5404-2f38-494d-a9be-4f55fe8cef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:24.134166 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca5404-2f38-494d-a9be-4f55fe8cef5d admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.134484 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca5404-2f38-494d-a9be-4f55fe8cef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.134827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca5404-2f38-494d-a9be-4f55fe8cef5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.135669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca5404-2f38-494d-a9be-4f55fe8cef5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:24.136001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca5404-2f38-494d-a9be-4f55fe8cef5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:24.137863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca5404-2f38-494d-a9be-4f55fe8cef5d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:24.154111 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1073/2111] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:24.155522 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6d432-fd88-4b9f-9675-b899503426e4 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:24.169023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6d432-fd88-4b9f-9675-b899503426e4 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:24.173773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176102e0-2390-4865-a216-2fcf777ece8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:24.174422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176102e0-2390-4865-a216-2fcf777ece8c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.174701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176102e0-2390-4865-a216-2fcf777ece8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.174978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176102e0-2390-4865-a216-2fcf777ece8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.175670 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176102e0-2390-4865-a216-2fcf777ece8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:24.177716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176102e0-2390-4865-a216-2fcf777ece8c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:24.181096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca5404-2f38-494d-a9be-4f55fe8cef5d admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:24.182968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9cca5404-2f38-494d-a9be-4f55fe8cef5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:24.207330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca5404-2f38-494d-a9be-4f55fe8cef5d admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:24.212838 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1039/2112] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:24 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:24.214751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176102e0-2390-4865-a216-2fcf777ece8c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:24.278987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:24.312200 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176102e0-2390-4865-a216-2fcf777ece8c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:24.314643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:24.316496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.317023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.317368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] PATH_INFO: `/v3/projects/0333c3e7e7a947909970b8857630955c` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.318732 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0333c3e7e7a947909970b8857630955c)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:24.319430 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c57b3-bceb-49df-8142-4ca10343b969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:24.320330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c57b3-bceb-49df-8142-4ca10343b969 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.321019 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c57b3-bceb-49df-8142-4ca10343b969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.321180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:24.321697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c57b3-bceb-49df-8142-4ca10343b969 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.322756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c57b3-bceb-49df-8142-4ca10343b969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:24.326091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c57b3-bceb-49df-8142-4ca10343b969 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:24.366688 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1040/2113] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:24 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:24.375067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c57b3-bceb-49df-8142-4ca10343b969 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:24.421511 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c57b3-bceb-49df-8142-4ca10343b969 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:24.433928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0333c3e7e7a947909970b8857630955c', 'request_id': 'req-44eedff7-3916-4042-a98e-1d2ca5d96fb9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:24.446673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0333c3e7e7a947909970b8857630955c', 'request_id': 'req-44eedff7-3916-4042-a98e-1d2ca5d96fb9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:24.463239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0333c3e7e7a947909970b8857630955c', 'request_id': 'req-44eedff7-3916-4042-a98e-1d2ca5d96fb9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:24.486919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] The token cache is being invalidate because project 0333c3e7e7a947909970b8857630955c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:24.487208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:24.488901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-44eedff7-3916-4042-a98e-1d2ca5d96fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:24.496284 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1074/2114] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:24 2022] DELETE /identity/v3/projects/0333c3e7e7a947909970b8857630955c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:24.500156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50440-5299-4a44-b7d7-8bf40fabd43c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.500446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50440-5299-4a44-b7d7-8bf40fabd43c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.500722 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50440-5299-4a44-b7d7-8bf40fabd43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.536404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-fea50440-5299-4a44-b7d7-8bf40fabd43c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:24.542470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-fea50440-5299-4a44-b7d7-8bf40fabd43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:24.558901 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815d923-740b-477c-a6f4-6ac3e6fd17f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:24.559615 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815d923-740b-477c-a6f4-6ac3e6fd17f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.559902 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815d923-740b-477c-a6f4-6ac3e6fd17f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.560186 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815d923-740b-477c-a6f4-6ac3e6fd17f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.563967 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815d923-740b-477c-a6f4-6ac3e6fd17f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:24.565673 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815d923-740b-477c-a6f4-6ac3e6fd17f8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:24.587203 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1041/2115] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:24.589744 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815d923-740b-477c-a6f4-6ac3e6fd17f8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:24.596832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8815d923-740b-477c-a6f4-6ac3e6fd17f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:24.628892 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:24.630184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815d923-740b-477c-a6f4-6ac3e6fd17f8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:24.646981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b8c26-c0b2-41cb-b0ea-8266b158fd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:24.647670 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b8c26-c0b2-41cb-b0ea-8266b158fd80 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.647950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b8c26-c0b2-41cb-b0ea-8266b158fd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.648228 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b8c26-c0b2-41cb-b0ea-8266b158fd80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.649122 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b8c26-c0b2-41cb-b0ea-8266b158fd80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:24.652075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b8c26-c0b2-41cb-b0ea-8266b158fd80 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:24.675475 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1042/2116] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:24.679822 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b8c26-c0b2-41cb-b0ea-8266b158fd80 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:24.687051 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1075/2117] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:24.698019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b20a86-a78d-4f1e-b174-734f214adfaf None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.698348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b20a86-a78d-4f1e-b174-734f214adfaf None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.698705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b20a86-a78d-4f1e-b174-734f214adfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.711738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b8c26-c0b2-41cb-b0ea-8266b158fd80 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:24.724682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6b511-d2a7-46e9-920f-538afa8a28e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:24.726589 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b511-d2a7-46e9-920f-538afa8a28e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.726876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b511-d2a7-46e9-920f-538afa8a28e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.728642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b511-d2a7-46e9-920f-538afa8a28e0 admin admin] PATH_INFO: `/v3/projects/2fa3fef1dd9e428e9e13c686a84836f9/users/454a5322c0b84e9190da70b3bd5d8f22/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.745957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-87b20a86-a78d-4f1e-b174-734f214adfaf None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:24.748492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-87b20a86-a78d-4f1e-b174-734f214adfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:24.763813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6b511-d2a7-46e9-920f-538afa8a28e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa3fef1dd9e428e9e13c686a84836f9, user_id=454a5322c0b84e9190da70b3bd5d8f22, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:24.766547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6b511-d2a7-46e9-920f-538afa8a28e0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:24.776895 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1076/2118] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:24.781575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b20a86-a78d-4f1e-b174-734f214adfaf None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:24.792400 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1043/2119] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:24 2022] PUT /identity/v3/projects/2fa3fef1dd9e428e9e13c686a84836f9/users/454a5322c0b84e9190da70b3bd5d8f22/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:24.800045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b20a86-a78d-4f1e-b174-734f214adfaf None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:24.803916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a1ac5-7648-4f75-b48b-5d376a6e04b0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.804201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a1ac5-7648-4f75-b48b-5d376a6e04b0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.804495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a1ac5-7648-4f75-b48b-5d376a6e04b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.870323 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cd8a1ac5-7648-4f75-b48b-5d376a6e04b0 None None] MFA Rules not processed for user `454a5322c0b84e9190da70b3bd5d8f22`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:24.872083 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cd8a1ac5-7648-4f75-b48b-5d376a6e04b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:24.938836 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d93af-9219-4609-b459-dfc5b27c730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:24.939577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d93af-9219-4609-b459-dfc5b27c730f admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.939858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d93af-9219-4609-b459-dfc5b27c730f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.940133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d93af-9219-4609-b459-dfc5b27c730f admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.941593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d93af-9219-4609-b459-dfc5b27c730f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:24.942053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d93af-9219-4609-b459-dfc5b27c730f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:24.943979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d93af-9219-4609-b459-dfc5b27c730f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:24.960440 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1077/2120] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:13:24.961313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d93af-9219-4609-b459-dfc5b27c730f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:24.963311 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5b4d93af-9219-4609-b459-dfc5b27c730f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:24.976613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d93af-9219-4609-b459-dfc5b27c730f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:24.981768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:24.982902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:24.983186 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:24.983590 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] PATH_INFO: `/v3/projects/16ee8fd269bd4879aeb6ca2a1910ed3a` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:24.984932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16ee8fd269bd4879aeb6ca2a1910ed3a)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:24.987369 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:25.020875 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1044/2121] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:25.022388 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a1ac5-7648-4f75-b48b-5d376a6e04b0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:25.037286 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a1ac5-7648-4f75-b48b-5d376a6e04b0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:25.049050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b1e4a-3fe4-461d-ae88-620abcb191d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:25.049947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b1e4a-3fe4-461d-ae88-620abcb191d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:25.050449 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b1e4a-3fe4-461d-ae88-620abcb191d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:25.051036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b1e4a-3fe4-461d-ae88-620abcb191d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:25.052115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b1e4a-3fe4-461d-ae88-620abcb191d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:25.054702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b1e4a-3fe4-461d-ae88-620abcb191d1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:25.056561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16ee8fd269bd4879aeb6ca2a1910ed3a', 'request_id': 'req-ca0d21bb-5f4d-4443-8330-da9e05fcc422'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.062324 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16ee8fd269bd4879aeb6ca2a1910ed3a', 'request_id': 'req-ca0d21bb-5f4d-4443-8330-da9e05fcc422'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.071519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16ee8fd269bd4879aeb6ca2a1910ed3a', 'request_id': 'req-ca0d21bb-5f4d-4443-8330-da9e05fcc422'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.077107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:25.088014 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] The token cache is being invalidate because project 16ee8fd269bd4879aeb6ca2a1910ed3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:25.088712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.089594 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.091944 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1078/2122] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:24 2022] DELETE /identity/v3/projects/16ee8fd269bd4879aeb6ca2a1910ed3a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:25.092736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:25.094048 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:25.096609 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1045/2123] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:25 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:25.097648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b1e4a-3fe4-461d-ae88-620abcb191d1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:25.099271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3b7b1e4a-3fe4-461d-ae88-620abcb191d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:25.161971 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d21bb-5f4d-4443-8330-da9e05fcc422 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:25.166028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:25.174748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b1e4a-3fe4-461d-ae88-620abcb191d1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:25.179811 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:25.181036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:25.181330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:25.181608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] PATH_INFO: `/v3/users/6bb3f5c4b1584b52871c7aa66241834b` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:25.182924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb3f5c4b1584b52871c7aa66241834b)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:25.185037 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:25.227221 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c0793-688b-4095-8a99-536de543b3bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:25.227757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0793-688b-4095-8a99-536de543b3bd service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:25.228077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0793-688b-4095-8a99-536de543b3bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:25.228287 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0793-688b-4095-8a99-536de543b3bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:25.230341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-832c0793-688b-4095-8a99-536de543b3bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:25.251335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb3f5c4b1584b52871c7aa66241834b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.252133 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb3f5c4b1584b52871c7aa66241834b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.252785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb3f5c4b1584b52871c7aa66241834b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.261686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb3f5c4b1584b52871c7aa66241834b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.275082 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb3f5c4b1584b52871c7aa66241834b'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.290641 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1046/2124] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:25 2022] DELETE /identity/v3/users/6bb3f5c4b1584b52871c7aa66241834b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:25.291861 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:25.293441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:25.305967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c0793-688b-4095-8a99-536de543b3bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:25.307834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c0793-688b-4095-8a99-536de543b3bd service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:25.319238 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1079/2125] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:25.408702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef57a2-83eb-430a-8665-7df24d71be36 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:25.413062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b23d88-35b7-4e60-8ef6-962643644ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:25.413685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23d88-35b7-4e60-8ef6-962643644ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:25.413966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23d88-35b7-4e60-8ef6-962643644ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:25.414262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23d88-35b7-4e60-8ef6-962643644ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:25.414964 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b23d88-35b7-4e60-8ef6-962643644ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:25.416932 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b23d88-35b7-4e60-8ef6-962643644ae9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:25.442433 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1047/2126] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:25 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:25.443600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c0793-688b-4095-8a99-536de543b3bd service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:25.445051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-832c0793-688b-4095-8a99-536de543b3bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:25.462198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c0793-688b-4095-8a99-536de543b3bd service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:25.470074 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be261a6-b005-406e-a3a2-b38644f37180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:25.471431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be261a6-b005-406e-a3a2-b38644f37180 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:25.471723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be261a6-b005-406e-a3a2-b38644f37180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:25.472200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be261a6-b005-406e-a3a2-b38644f37180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:25.473197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be261a6-b005-406e-a3a2-b38644f37180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:25.474764 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be261a6-b005-406e-a3a2-b38644f37180 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:25.508416 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1080/2127] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:25.512996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b23d88-35b7-4e60-8ef6-962643644ae9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:25.520978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:25.526167 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b23d88-35b7-4e60-8ef6-962643644ae9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:25.538140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc702a-382a-4436-8945-b1a58738be4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:25.539868 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc702a-382a-4436-8945-b1a58738be4a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:25.540160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc702a-382a-4436-8945-b1a58738be4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:25.541070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc702a-382a-4436-8945-b1a58738be4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:25.542297 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc702a-382a-4436-8945-b1a58738be4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:25.543943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc702a-382a-4436-8945-b1a58738be4a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:25.560180 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1048/2128] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:25.564373 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be261a6-b005-406e-a3a2-b38644f37180 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:25.580961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be261a6-b005-406e-a3a2-b38644f37180 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:25.588794 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecd642-3be7-4a46-bdc3-1c196816077f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:25.589783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd642-3be7-4a46-bdc3-1c196816077f admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:25.590069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd642-3be7-4a46-bdc3-1c196816077f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:25.590566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd642-3be7-4a46-bdc3-1c196816077f admin admin] PATH_INFO: `/v3/projects/befdb4e46e9a4620a2855ab29a700d90/users/7fe727cbd50d4aa3adeb22d7a469848d/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:25.636301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecd642-3be7-4a46-bdc3-1c196816077f admin admin] RBAC: Authorizing `identity:create_grant(project_id=befdb4e46e9a4620a2855ab29a700d90, user_id=7fe727cbd50d4aa3adeb22d7a469848d, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:25.646236 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecd642-3be7-4a46-bdc3-1c196816077f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:25.654533 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc702a-382a-4436-8945-b1a58738be4a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:25.656804 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e1dc702a-382a-4436-8945-b1a58738be4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:25.672729 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1081/2129] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:25 2022] PUT /identity/v3/projects/befdb4e46e9a4620a2855ab29a700d90/users/7fe727cbd50d4aa3adeb22d7a469848d/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:25.675475 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc702a-382a-4436-8945-b1a58738be4a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:25.683396 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb771b0-fed9-4e58-a64d-e1269ea07c09 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:25.683683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb771b0-fed9-4e58-a64d-e1269ea07c09 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:25.683953 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb771b0-fed9-4e58-a64d-e1269ea07c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:25.731310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-cdb771b0-fed9-4e58-a64d-e1269ea07c09 None None] MFA Rules not processed for user `7fe727cbd50d4aa3adeb22d7a469848d`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:25.731310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-cdb771b0-fed9-4e58-a64d-e1269ea07c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:25.772110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:25.773270 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:25.778562 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:25.778966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] PATH_INFO: `/v3/projects/fc71b981cab0426b9190331efea714c7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:25.780373 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc71b981cab0426b9190331efea714c7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:25.785093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:25.802658 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1082/2130] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:25.807261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb771b0-fed9-4e58-a64d-e1269ea07c09 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:25.817975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb771b0-fed9-4e58-a64d-e1269ea07c09 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:25.823223 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98468a-a73a-4799-822f-8f0b079b4eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:25.823903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98468a-a73a-4799-822f-8f0b079b4eac admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:25.824241 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98468a-a73a-4799-822f-8f0b079b4eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:25.824499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98468a-a73a-4799-822f-8f0b079b4eac admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:25.825229 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98468a-a73a-4799-822f-8f0b079b4eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:25.827226 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98468a-a73a-4799-822f-8f0b079b4eac admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:25.846173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc71b981cab0426b9190331efea714c7', 'request_id': 'req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.853417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:25.859308 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc71b981cab0426b9190331efea714c7', 'request_id': 'req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.868792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc71b981cab0426b9190331efea714c7', 'request_id': 'req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.884073 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1083/2131] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:25 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:25.884376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] The token cache is being invalidate because project fc71b981cab0426b9190331efea714c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:25.884376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.884850 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f7e95c40-4fe8-49d2-b153-76438f4a5cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:25.890200 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1049/2132] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:25 2022] DELETE /identity/v3/projects/fc71b981cab0426b9190331efea714c7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:25.890461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98468a-a73a-4799-822f-8f0b079b4eac admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:25.892411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5710-c165-4bc3-9bd4-e0bcea9e26d3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:25.892685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5710-c165-4bc3-9bd4-e0bcea9e26d3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:25.893094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5710-c165-4bc3-9bd4-e0bcea9e26d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:25.897116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7f98468a-a73a-4799-822f-8f0b079b4eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:25.916865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-521d5710-c165-4bc3-9bd4-e0bcea9e26d3 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:25.918259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-521d5710-c165-4bc3-9bd4-e0bcea9e26d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:25.992474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98468a-a73a-4799-822f-8f0b079b4eac admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.003767 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1050/2133] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:26.003767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0303d-ad74-4ce7-b955-78760646cfb6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.003767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0303d-ad74-4ce7-b955-78760646cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.003767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0303d-ad74-4ce7-b955-78760646cfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.008863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17dd4b7-7054-4327-8fe0-a0eeaaf9feab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.009895 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dd4b7-7054-4327-8fe0-a0eeaaf9feab admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.010484 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dd4b7-7054-4327-8fe0-a0eeaaf9feab admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.011141 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dd4b7-7054-4327-8fe0-a0eeaaf9feab admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.012160 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17dd4b7-7054-4327-8fe0-a0eeaaf9feab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.016458 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17dd4b7-7054-4327-8fe0-a0eeaaf9feab admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.023093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-65f0303d-ad74-4ce7-b955-78760646cfb6 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:26.024330 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-65f0303d-ad74-4ce7-b955-78760646cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:26.032588 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1051/2134] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:26.044055 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0303d-ad74-4ce7-b955-78760646cfb6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.057284 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1084/2135] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:25 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:26.059143 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17dd4b7-7054-4327-8fe0-a0eeaaf9feab admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.059995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0303d-ad74-4ce7-b955-78760646cfb6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.065021 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01659da4-04a6-4ce2-9cee-df5a36c11d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.065792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01659da4-04a6-4ce2-9cee-df5a36c11d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.066106 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01659da4-04a6-4ce2-9cee-df5a36c11d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.066440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01659da4-04a6-4ce2-9cee-df5a36c11d91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.067206 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01659da4-04a6-4ce2-9cee-df5a36c11d91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:26.067954 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01659da4-04a6-4ce2-9cee-df5a36c11d91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.069601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01659da4-04a6-4ce2-9cee-df5a36c11d91 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.076151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17dd4b7-7054-4327-8fe0-a0eeaaf9feab admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.081861 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c123-4287-416e-84ca-0858b464c79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.082691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c123-4287-416e-84ca-0858b464c79d admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.083089 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c123-4287-416e-84ca-0858b464c79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.083456 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c123-4287-416e-84ca-0858b464c79d admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.084745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380c123-4287-416e-84ca-0858b464c79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.086673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380c123-4287-416e-84ca-0858b464c79d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.087109 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1052/2136] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:26.088117 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01659da4-04a6-4ce2-9cee-df5a36c11d91 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.095380 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:26.100516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01659da4-04a6-4ce2-9cee-df5a36c11d91 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.104646 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1085/2137] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:26.107127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c123-4287-416e-84ca-0858b464c79d admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.110603 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0014697-b798-4037-9b07-4f7ddda55bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.111292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014697-b798-4037-9b07-4f7ddda55bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.111581 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014697-b798-4037-9b07-4f7ddda55bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.111861 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0014697-b798-4037-9b07-4f7ddda55bc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.112609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0014697-b798-4037-9b07-4f7ddda55bc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.115205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0014697-b798-4037-9b07-4f7ddda55bc7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.122851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c123-4287-416e-84ca-0858b464c79d admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.128700 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb895f3-b700-4cd0-be18-98049cdd41b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.129745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb895f3-b700-4cd0-be18-98049cdd41b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.130143 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb895f3-b700-4cd0-be18-98049cdd41b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.130545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb895f3-b700-4cd0-be18-98049cdd41b1 admin admin] PATH_INFO: `/v3/projects/360043ee37b7421494e1f070d9331006/users/2de36ab957ab4ca4aa2f327bfbf1c013/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.150583 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1053/2138] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:26 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:26.150882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb895f3-b700-4cd0-be18-98049cdd41b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=360043ee37b7421494e1f070d9331006, user_id=2de36ab957ab4ca4aa2f327bfbf1c013, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.151651 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb895f3-b700-4cd0-be18-98049cdd41b1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.151805 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0014697-b798-4037-9b07-4f7ddda55bc7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.169829 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1086/2139] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:26 2022] PUT /identity/v3/projects/360043ee37b7421494e1f070d9331006/users/2de36ab957ab4ca4aa2f327bfbf1c013/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:26.185593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb895f3-b700-4cd0-be18-98049cdd41b1 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.254520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0014697-b798-4037-9b07-4f7ddda55bc7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.269250 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb895f3-b700-4cd0-be18-98049cdd41b1 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.278318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b3f50-2dad-4190-8e3a-d25df0314f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.279256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b3f50-2dad-4190-8e3a-d25df0314f17 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.279731 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b3f50-2dad-4190-8e3a-d25df0314f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.280232 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b3f50-2dad-4190-8e3a-d25df0314f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.281165 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b3f50-2dad-4190-8e3a-d25df0314f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.283450 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b3f50-2dad-4190-8e3a-d25df0314f17 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.295358 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1087/2140] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:26.299796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b3f50-2dad-4190-8e3a-d25df0314f17 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.310029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b3f50-2dad-4190-8e3a-d25df0314f17 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.314681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1da61-e4af-4d5b-9dc3-92e385cb3f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.315620 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1da61-e4af-4d5b-9dc3-92e385cb3f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.315897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1da61-e4af-4d5b-9dc3-92e385cb3f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.316168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1da61-e4af-4d5b-9dc3-92e385cb3f20 admin admin] PATH_INFO: `/v3/projects/360043ee37b7421494e1f070d9331006/users/2de36ab957ab4ca4aa2f327bfbf1c013/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.318164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1da61-e4af-4d5b-9dc3-92e385cb3f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=360043ee37b7421494e1f070d9331006, user_id=2de36ab957ab4ca4aa2f327bfbf1c013, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.320857 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1da61-e4af-4d5b-9dc3-92e385cb3f20 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.326334 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7bf72-387e-4b9c-b873-88c88e8e08c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.327064 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7bf72-387e-4b9c-b873-88c88e8e08c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.327426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7bf72-387e-4b9c-b873-88c88e8e08c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.327704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7bf72-387e-4b9c-b873-88c88e8e08c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.328390 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7bf72-387e-4b9c-b873-88c88e8e08c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.330327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7bf72-387e-4b9c-b873-88c88e8e08c2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.331353 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1088/2141] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:26 2022] PUT /identity/v3/projects/360043ee37b7421494e1f070d9331006/users/2de36ab957ab4ca4aa2f327bfbf1c013/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:26.343090 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcba197-e15d-4e2a-b174-cfa7d5ce4f7e None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.343443 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcba197-e15d-4e2a-b174-cfa7d5ce4f7e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.343819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcba197-e15d-4e2a-b174-cfa7d5ce4f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.353924 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1054/2142] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:26 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:26.357801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7bf72-387e-4b9c-b873-88c88e8e08c2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.368741 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7bf72-387e-4b9c-b873-88c88e8e08c2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.371425 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-1bcba197-e15d-4e2a-b174-cfa7d5ce4f7e None None] MFA Rules not processed for user `2de36ab957ab4ca4aa2f327bfbf1c013`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:26.372770 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1bcba197-e15d-4e2a-b174-cfa7d5ce4f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:26.392379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:26.437118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337642bf-a517-4b46-b9f8-bc424aade01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.437859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337642bf-a517-4b46-b9f8-bc424aade01c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.438135 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337642bf-a517-4b46-b9f8-bc424aade01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.438451 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337642bf-a517-4b46-b9f8-bc424aade01c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.439193 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337642bf-a517-4b46-b9f8-bc424aade01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.440799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337642bf-a517-4b46-b9f8-bc424aade01c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.454818 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1089/2143] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:26.455049 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1055/2144] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:26.459262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcba197-e15d-4e2a-b174-cfa7d5ce4f7e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.466493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337642bf-a517-4b46-b9f8-bc424aade01c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.468479 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-337642bf-a517-4b46-b9f8-bc424aade01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:26.471118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcba197-e15d-4e2a-b174-cfa7d5ce4f7e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.476133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16bd8d-f431-4a53-863e-6748c12fb0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.476834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16bd8d-f431-4a53-863e-6748c12fb0ed admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.477115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16bd8d-f431-4a53-863e-6748c12fb0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.477390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16bd8d-f431-4a53-863e-6748c12fb0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.478132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16bd8d-f431-4a53-863e-6748c12fb0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.479952 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16bd8d-f431-4a53-863e-6748c12fb0ed admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.490607 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1090/2145] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:26.494487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16bd8d-f431-4a53-863e-6748c12fb0ed admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.495672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:26.506027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16bd8d-f431-4a53-863e-6748c12fb0ed admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.512039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3a1db-9426-43e0-aa18-eecb5a6289eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.513128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3a1db-9426-43e0-aa18-eecb5a6289eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.513433 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3a1db-9426-43e0-aa18-eecb5a6289eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.513729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3a1db-9426-43e0-aa18-eecb5a6289eb admin admin] PATH_INFO: `/v3/projects/d018aa73d69e4dae8fb434ed1c87cdc4/users/cc3268d3f1244eb29a382bcca3d62065/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.536438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3a1db-9426-43e0-aa18-eecb5a6289eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d018aa73d69e4dae8fb434ed1c87cdc4, user_id=cc3268d3f1244eb29a382bcca3d62065, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.539485 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3a1db-9426-43e0-aa18-eecb5a6289eb admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.543683 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337642bf-a517-4b46-b9f8-bc424aade01c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.547386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377d8ee-ce85-45df-bd06-d4ee5c630229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.548047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377d8ee-ce85-45df-bd06-d4ee5c630229 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.548350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377d8ee-ce85-45df-bd06-d4ee5c630229 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.548638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377d8ee-ce85-45df-bd06-d4ee5c630229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.558105 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1091/2146] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:26 2022] PUT /identity/v3/projects/d018aa73d69e4dae8fb434ed1c87cdc4/users/cc3268d3f1244eb29a382bcca3d62065/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:26.558328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377d8ee-ce85-45df-bd06-d4ee5c630229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.559693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377d8ee-ce85-45df-bd06-d4ee5c630229 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.567977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61651e-53fc-416d-8863-7c6146891041 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.568264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61651e-53fc-416d-8863-7c6146891041 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.568537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61651e-53fc-416d-8863-7c6146891041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.569965 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1056/2147] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:26 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:26.600304 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ec61651e-53fc-416d-8863-7c6146891041 None None] MFA Rules not processed for user `cc3268d3f1244eb29a382bcca3d62065`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:26.602062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ec61651e-53fc-416d-8863-7c6146891041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:26.674122 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1092/2148] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:26.678651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377d8ee-ce85-45df-bd06-d4ee5c630229 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.689763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377d8ee-ce85-45df-bd06-d4ee5c630229 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.766559 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaff84d-7418-4b35-9cc1-d983b2f44340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.766559 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaff84d-7418-4b35-9cc1-d983b2f44340 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.766559 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaff84d-7418-4b35-9cc1-d983b2f44340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.766559 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaff84d-7418-4b35-9cc1-d983b2f44340 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.767652 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaff84d-7418-4b35-9cc1-d983b2f44340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.768420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaff84d-7418-4b35-9cc1-d983b2f44340 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.818260 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1057/2149] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:26 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:26.820614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61651e-53fc-416d-8863-7c6146891041 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.834313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61651e-53fc-416d-8863-7c6146891041 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.912734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518cc58-425d-4dcb-9fab-cc8f455b7cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.914280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518cc58-425d-4dcb-9fab-cc8f455b7cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.914280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518cc58-425d-4dcb-9fab-cc8f455b7cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.914280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518cc58-425d-4dcb-9fab-cc8f455b7cf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.915048 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518cc58-425d-4dcb-9fab-cc8f455b7cf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.917169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518cc58-425d-4dcb-9fab-cc8f455b7cf9 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.942435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:26.951248 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1093/2150] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:26 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:26.954860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaff84d-7418-4b35-9cc1-d983b2f44340 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:26.967918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaff84d-7418-4b35-9cc1-d983b2f44340 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:26.976359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d5f58-7c06-4310-b5e4-0bcb8e60cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:26.977054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d5f58-7c06-4310-b5e4-0bcb8e60cc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:26.977336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d5f58-7c06-4310-b5e4-0bcb8e60cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:26.977604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d5f58-7c06-4310-b5e4-0bcb8e60cc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:26.978521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d5f58-7c06-4310-b5e4-0bcb8e60cc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:26.980304 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d5f58-7c06-4310-b5e4-0bcb8e60cc03 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:26.998995 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1058/2151] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:27.003545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518cc58-425d-4dcb-9fab-cc8f455b7cf9 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:27.035001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518cc58-425d-4dcb-9fab-cc8f455b7cf9 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:27.049826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e5e8b-ee17-49b6-a1f7-3eeb8fa22d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:27.051313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e5e8b-ee17-49b6-a1f7-3eeb8fa22d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:27.051802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e5e8b-ee17-49b6-a1f7-3eeb8fa22d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:27.052284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e5e8b-ee17-49b6-a1f7-3eeb8fa22d53 admin admin] PATH_INFO: `/v3/projects/0453aba57f6444379d4f2fddcb60d1f7/users/3c7a70aa53c04aabaf0e32e8489f7c5a/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:27.071085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d5f58-7c06-4310-b5e4-0bcb8e60cc03 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:27.093033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d5f58-7c06-4310-b5e4-0bcb8e60cc03 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:27.094290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e5e8b-ee17-49b6-a1f7-3eeb8fa22d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0453aba57f6444379d4f2fddcb60d1f7, user_id=3c7a70aa53c04aabaf0e32e8489f7c5a, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:27.098200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e5e8b-ee17-49b6-a1f7-3eeb8fa22d53 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:27.108976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabac1bb-c282-4198-bc55-26ad6f6ae269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:27.109990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabac1bb-c282-4198-bc55-26ad6f6ae269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:27.110280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabac1bb-c282-4198-bc55-26ad6f6ae269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:27.110720 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabac1bb-c282-4198-bc55-26ad6f6ae269 admin admin] PATH_INFO: `/v3/users/7fe727cbd50d4aa3adeb22d7a469848d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:27.113634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabac1bb-c282-4198-bc55-26ad6f6ae269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fe727cbd50d4aa3adeb22d7a469848d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:27.119664 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabac1bb-c282-4198-bc55-26ad6f6ae269 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:27.123071 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1094/2152] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:27 2022] PUT /identity/v3/projects/0453aba57f6444379d4f2fddcb60d1f7/users/3c7a70aa53c04aabaf0e32e8489f7c5a/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:27.128010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e5e8b-ee17-49b6-a1f7-3eeb8fa22d53 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:27.189144 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dabac1bb-c282-4198-bc55-26ad6f6ae269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fe727cbd50d4aa3adeb22d7a469848d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:27.190023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dabac1bb-c282-4198-bc55-26ad6f6ae269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fe727cbd50d4aa3adeb22d7a469848d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:27.190674 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dabac1bb-c282-4198-bc55-26ad6f6ae269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fe727cbd50d4aa3adeb22d7a469848d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:27.219678 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e5e8b-ee17-49b6-a1f7-3eeb8fa22d53 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:27.223814 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dabac1bb-c282-4198-bc55-26ad6f6ae269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fe727cbd50d4aa3adeb22d7a469848d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:27.237557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b10fe2-2e3d-4c69-ae9a-259db6c4d3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:27.237557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10fe2-2e3d-4c69-ae9a-259db6c4d3de admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:27.237880 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10fe2-2e3d-4c69-ae9a-259db6c4d3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:27.238147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10fe2-2e3d-4c69-ae9a-259db6c4d3de admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:27.238946 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b10fe2-2e3d-4c69-ae9a-259db6c4d3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:27.240604 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b10fe2-2e3d-4c69-ae9a-259db6c4d3de admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:27.243433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-dabac1bb-c282-4198-bc55-26ad6f6ae269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fe727cbd50d4aa3adeb22d7a469848d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:27.264876 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1095/2153] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:27.270590 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1059/2154] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:27 2022] DELETE /identity/v3/users/7fe727cbd50d4aa3adeb22d7a469848d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:27.270749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b10fe2-2e3d-4c69-ae9a-259db6c4d3de admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:27.271965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-45b10fe2-2e3d-4c69-ae9a-259db6c4d3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:27.272084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517acaa-2308-4a7a-9302-2f65291974d5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:27.272283 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517acaa-2308-4a7a-9302-2f65291974d5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:27.272562 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517acaa-2308-4a7a-9302-2f65291974d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:27.280107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:27.296432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b517acaa-2308-4a7a-9302-2f65291974d5 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:27.298508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b517acaa-2308-4a7a-9302-2f65291974d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:27.300045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b10fe2-2e3d-4c69-ae9a-259db6c4d3de admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:27.368179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c17da-4cc6-4457-b2cb-606255ebff4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:27.369413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c17da-4cc6-4457-b2cb-606255ebff4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:27.369985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c17da-4cc6-4457-b2cb-606255ebff4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:27.369985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c17da-4cc6-4457-b2cb-606255ebff4f admin admin] PATH_INFO: `/v3/projects/0453aba57f6444379d4f2fddcb60d1f7/users/3c7a70aa53c04aabaf0e32e8489f7c5a/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:27.372665 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c17da-4cc6-4457-b2cb-606255ebff4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0453aba57f6444379d4f2fddcb60d1f7, user_id=3c7a70aa53c04aabaf0e32e8489f7c5a, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:27.374269 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1060/2155] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:27.375599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c17da-4cc6-4457-b2cb-606255ebff4f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:27.383382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517acaa-2308-4a7a-9302-2f65291974d5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:27.385248 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b517acaa-2308-4a7a-9302-2f65291974d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:27.388233 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1096/2156] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:27 2022] PUT /identity/v3/projects/0453aba57f6444379d4f2fddcb60d1f7/users/3c7a70aa53c04aabaf0e32e8489f7c5a/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:27.399664 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192aedf0-1cfe-4b5d-8f9d-d337c579cda4 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:27.399955 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192aedf0-1cfe-4b5d-8f9d-d337c579cda4 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:27.400230 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192aedf0-1cfe-4b5d-8f9d-d337c579cda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:27.430116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-192aedf0-1cfe-4b5d-8f9d-d337c579cda4 None None] MFA Rules not processed for user `3c7a70aa53c04aabaf0e32e8489f7c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:27.433024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-192aedf0-1cfe-4b5d-8f9d-d337c579cda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:27.467900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517acaa-2308-4a7a-9302-2f65291974d5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:27.495246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:27.516814 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1097/2157] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:27.522339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ad758-838b-45fd-af6f-ffb9e41292e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:27.522902 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ad758-838b-45fd-af6f-ffb9e41292e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:27.523183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ad758-838b-45fd-af6f-ffb9e41292e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:27.523455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ad758-838b-45fd-af6f-ffb9e41292e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:27.527561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192aedf0-1cfe-4b5d-8f9d-d337c579cda4 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:27.529141 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-192aedf0-1cfe-4b5d-8f9d-d337c579cda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:27.583485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ad758-838b-45fd-af6f-ffb9e41292e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:27.585295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ad758-838b-45fd-af6f-ffb9e41292e9 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:27.586181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192aedf0-1cfe-4b5d-8f9d-d337c579cda4 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:27.590028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469df4b9-ed96-4c45-845e-62e92f426943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:27.590722 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469df4b9-ed96-4c45-845e-62e92f426943 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:27.591004 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469df4b9-ed96-4c45-845e-62e92f426943 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:27.591283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469df4b9-ed96-4c45-845e-62e92f426943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:27.594935 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1061/2158] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:27.601130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469df4b9-ed96-4c45-845e-62e92f426943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:27.603707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469df4b9-ed96-4c45-845e-62e92f426943 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:27.616170 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1098/2159] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:27.935677 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ad758-838b-45fd-af6f-ffb9e41292e9 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:27.937107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-049ad758-838b-45fd-af6f-ffb9e41292e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:27.953424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ad758-838b-45fd-af6f-ffb9e41292e9 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:27.962377 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:27.963331 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:27.963623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:27.963897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] PATH_INFO: `/v3/projects/befdb4e46e9a4620a2855ab29a700d90` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:27.965126 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=befdb4e46e9a4620a2855ab29a700d90)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:27.967083 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:27.979347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:28.028422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'befdb4e46e9a4620a2855ab29a700d90', 'request_id': 'req-ec77e1fa-b269-4f4a-9172-953c3555c406'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.037492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'befdb4e46e9a4620a2855ab29a700d90', 'request_id': 'req-ec77e1fa-b269-4f4a-9172-953c3555c406'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.044732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'befdb4e46e9a4620a2855ab29a700d90', 'request_id': 'req-ec77e1fa-b269-4f4a-9172-953c3555c406'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.068847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469df4b9-ed96-4c45-845e-62e92f426943 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:28.069294 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] The token cache is being invalidate because project befdb4e46e9a4620a2855ab29a700d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:28.069845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.070627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.074259 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1062/2160] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:27 2022] DELETE /identity/v3/projects/befdb4e46e9a4620a2855ab29a700d90 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:28.077899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:28.083440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:28.099718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469df4b9-ed96-4c45-845e-62e92f426943 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:28.166872 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f79e52-cf32-4495-a0f0-42240bdb8ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:28.167877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f79e52-cf32-4495-a0f0-42240bdb8ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:28.168230 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f79e52-cf32-4495-a0f0-42240bdb8ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:28.168507 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f79e52-cf32-4495-a0f0-42240bdb8ff0 admin admin] PATH_INFO: `/v3/users/cc3268d3f1244eb29a382bcca3d62065` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:28.169735 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f79e52-cf32-4495-a0f0-42240bdb8ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc3268d3f1244eb29a382bcca3d62065)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:28.171855 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f79e52-cf32-4495-a0f0-42240bdb8ff0 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:28.172207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77e1fa-b269-4f4a-9172-953c3555c406 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:28.179147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:28.180173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:28.180462 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:28.180751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] PATH_INFO: `/v3/users/2de36ab957ab4ca4aa2f327bfbf1c013` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:28.183105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de36ab957ab4ca4aa2f327bfbf1c013)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:28.185382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:28.241415 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-45f79e52-cf32-4495-a0f0-42240bdb8ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc3268d3f1244eb29a382bcca3d62065'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.243903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-45f79e52-cf32-4495-a0f0-42240bdb8ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc3268d3f1244eb29a382bcca3d62065'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.272413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de36ab957ab4ca4aa2f327bfbf1c013'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.273821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de36ab957ab4ca4aa2f327bfbf1c013'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.274762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-45f79e52-cf32-4495-a0f0-42240bdb8ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc3268d3f1244eb29a382bcca3d62065'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.277276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de36ab957ab4ca4aa2f327bfbf1c013'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.289887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-45f79e52-cf32-4495-a0f0-42240bdb8ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc3268d3f1244eb29a382bcca3d62065'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.290095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de36ab957ab4ca4aa2f327bfbf1c013'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.293038 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-45f79e52-cf32-4495-a0f0-42240bdb8ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc3268d3f1244eb29a382bcca3d62065'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.319342 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1099/2161] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:28 2022] DELETE /identity/v3/users/cc3268d3f1244eb29a382bcca3d62065 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:28.324725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05c0ab-c3fb-4deb-986c-398ec8169d8f None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:28.325010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05c0ab-c3fb-4deb-986c-398ec8169d8f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:28.328022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de36ab957ab4ca4aa2f327bfbf1c013'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:28.353671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05c0ab-c3fb-4deb-986c-398ec8169d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:28.353808 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1063/2162] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:28 2022] DELETE /identity/v3/users/2de36ab957ab4ca4aa2f327bfbf1c013 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:28.380896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-7b05c0ab-c3fb-4deb-986c-398ec8169d8f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:28.382277 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7b05c0ab-c3fb-4deb-986c-398ec8169d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:28.395825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:28.463545 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1100/2163] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:28.481183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:28.483631 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:28.505406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:28.566053 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac274de7-924a-4f0d-85e7-3ed776c6de0e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:28.569885 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b7bd6-2f00-4bf4-a329-a5bbbe791bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:28.570656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b7bd6-2f00-4bf4-a329-a5bbbe791bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:28.571040 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b7bd6-2f00-4bf4-a329-a5bbbe791bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:28.571385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b7bd6-2f00-4bf4-a329-a5bbbe791bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:28.601013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b7bd6-2f00-4bf4-a329-a5bbbe791bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:28.602899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b7bd6-2f00-4bf4-a329-a5bbbe791bc7 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:28.617444 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1064/2164] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:28.898114 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05c0ab-c3fb-4deb-986c-398ec8169d8f None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:28.899525 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7b05c0ab-c3fb-4deb-986c-398ec8169d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:28.921986 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05c0ab-c3fb-4deb-986c-398ec8169d8f None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:28.935175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:28.936082 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:28.936367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:28.936641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] PATH_INFO: `/v3/projects/360043ee37b7421494e1f070d9331006` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:28.938969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=360043ee37b7421494e1f070d9331006)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:28.941565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:29.007465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '360043ee37b7421494e1f070d9331006', 'request_id': 'req-e0404bdd-d563-4928-8127-347146727225'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.013037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '360043ee37b7421494e1f070d9331006', 'request_id': 'req-e0404bdd-d563-4928-8127-347146727225'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.020914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '360043ee37b7421494e1f070d9331006', 'request_id': 'req-e0404bdd-d563-4928-8127-347146727225'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.036787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] The token cache is being invalidate because project 360043ee37b7421494e1f070d9331006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:29.037055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.037834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.039984 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1101/2165] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:28 2022] DELETE /identity/v3/projects/360043ee37b7421494e1f070d9331006 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:29.046664 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efbac7-37a2-40d4-b2f7-a2d25163ad24 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:29.047320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efbac7-37a2-40d4-b2f7-a2d25163ad24 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:29.047922 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efbac7-37a2-40d4-b2f7-a2d25163ad24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:29.073218 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-41efbac7-37a2-40d4-b2f7-a2d25163ad24 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:29.073218 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-41efbac7-37a2-40d4-b2f7-a2d25163ad24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:29.074811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:29.076785 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:29.148842 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1065/2166] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:29.156942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db8dec-46ba-41db-8049-5b564ab5fcf7 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:29.157400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db8dec-46ba-41db-8049-5b564ab5fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:29.157814 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db8dec-46ba-41db-8049-5b564ab5fcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:29.165811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0404bdd-d563-4928-8127-347146727225 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:29.171988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:29.173022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:29.173372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:29.174096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] PATH_INFO: `/v3/projects/d018aa73d69e4dae8fb434ed1c87cdc4` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:29.175351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d018aa73d69e4dae8fb434ed1c87cdc4)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:29.177588 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:29.180005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-83db8dec-46ba-41db-8049-5b564ab5fcf7 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:29.181668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-83db8dec-46ba-41db-8049-5b564ab5fcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:29.196371 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1066/2167] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:29.197935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db8dec-46ba-41db-8049-5b564ab5fcf7 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:29.212135 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db8dec-46ba-41db-8049-5b564ab5fcf7 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:29.218367 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c022409-6f3f-483e-be27-280b19875e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:29.219053 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022409-6f3f-483e-be27-280b19875e9b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:29.219381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022409-6f3f-483e-be27-280b19875e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:29.219655 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022409-6f3f-483e-be27-280b19875e9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:29.220440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c022409-6f3f-483e-be27-280b19875e9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:29.220749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c022409-6f3f-483e-be27-280b19875e9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:29.222628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c022409-6f3f-483e-be27-280b19875e9b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:29.232032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:29.237952 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1067/2168] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:29.243407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c022409-6f3f-483e-be27-280b19875e9b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:29.243650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd018aa73d69e4dae8fb434ed1c87cdc4', 'request_id': 'req-6a6dd82c-9589-4362-97a1-00c6d52fa49c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.251130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd018aa73d69e4dae8fb434ed1c87cdc4', 'request_id': 'req-6a6dd82c-9589-4362-97a1-00c6d52fa49c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.254944 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c022409-6f3f-483e-be27-280b19875e9b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:29.257868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd018aa73d69e4dae8fb434ed1c87cdc4', 'request_id': 'req-6a6dd82c-9589-4362-97a1-00c6d52fa49c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.276740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] The token cache is being invalidate because project d018aa73d69e4dae8fb434ed1c87cdc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:29.277328 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.278394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.280006 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1102/2169] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:29 2022] DELETE /identity/v3/projects/d018aa73d69e4dae8fb434ed1c87cdc4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:29.284322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:29.286115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:29.320762 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3a930-722e-4c19-9b33-1089589f40d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:29.321563 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3a930-722e-4c19-9b33-1089589f40d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:29.321835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3a930-722e-4c19-9b33-1089589f40d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:29.322116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3a930-722e-4c19-9b33-1089589f40d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:29.323044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3a930-722e-4c19-9b33-1089589f40d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:29.325049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3a930-722e-4c19-9b33-1089589f40d8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:29.355359 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1068/2170] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:29 2022] POST /identity/v3/projects => generated 391 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:29.359654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6dd82c-9589-4362-97a1-00c6d52fa49c admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:29.361173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3a930-722e-4c19-9b33-1089589f40d8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:29.362808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-74a3a930-722e-4c19-9b33-1089589f40d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:29.391379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:29.465929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:29.466869 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:29.467151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:29.467476 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] PATH_INFO: `/v3/users/3c7a70aa53c04aabaf0e32e8489f7c5a` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:29.468669 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3a930-722e-4c19-9b33-1089589f40d8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:29.472891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c7a70aa53c04aabaf0e32e8489f7c5a)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:29.475573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab4430-bb2d-43d6-9443-730bd0dd8665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:29.476523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab4430-bb2d-43d6-9443-730bd0dd8665 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:29.477001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab4430-bb2d-43d6-9443-730bd0dd8665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:29.477464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab4430-bb2d-43d6-9443-730bd0dd8665 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:29.478334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:29.479188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab4430-bb2d-43d6-9443-730bd0dd8665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:29.482408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab4430-bb2d-43d6-9443-730bd0dd8665 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:29.513791 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1069/2171] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:29 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:29.518012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab4430-bb2d-43d6-9443-730bd0dd8665 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:29.534716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab4430-bb2d-43d6-9443-730bd0dd8665 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:29.541626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e49bca-6d46-4b47-bf40-8e3f09020035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:29.542771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e49bca-6d46-4b47-bf40-8e3f09020035 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:29.542883 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e49bca-6d46-4b47-bf40-8e3f09020035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:29.543335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e49bca-6d46-4b47-bf40-8e3f09020035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:29.548847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e49bca-6d46-4b47-bf40-8e3f09020035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:29.551889 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e49bca-6d46-4b47-bf40-8e3f09020035 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:29.567860 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c7a70aa53c04aabaf0e32e8489f7c5a'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.568660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c7a70aa53c04aabaf0e32e8489f7c5a'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.574945 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1070/2172] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:29.577884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e49bca-6d46-4b47-bf40-8e3f09020035 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:29.583455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b2e49bca-6d46-4b47-bf40-8e3f09020035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:29.621688 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c7a70aa53c04aabaf0e32e8489f7c5a'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.629173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e49bca-6d46-4b47-bf40-8e3f09020035 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:29.638547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c7a70aa53c04aabaf0e32e8489f7c5a'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.641501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c7a70aa53c04aabaf0e32e8489f7c5a'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:29.644633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:29.645264 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac7b20-5d14-43fe-aca9-90e781370aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:29.645953 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac7b20-5d14-43fe-aca9-90e781370aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:29.648472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac7b20-5d14-43fe-aca9-90e781370aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:29.648782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac7b20-5d14-43fe-aca9-90e781370aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:29.649655 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac7b20-5d14-43fe-aca9-90e781370aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:29.652529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac7b20-5d14-43fe-aca9-90e781370aa9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:29.667000 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1103/2173] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:29 2022] DELETE /identity/v3/users/3c7a70aa53c04aabaf0e32e8489f7c5a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:29.678476 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1071/2174] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:29.681193 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:29.790012 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508883a-2067-43dd-9254-4402b3aa5f7f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:29.795772 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0a048-10b1-4729-87f8-978c7c97a953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:29.796728 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0a048-10b1-4729-87f8-978c7c97a953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:29.797010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0a048-10b1-4729-87f8-978c7c97a953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:29.797630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0a048-10b1-4729-87f8-978c7c97a953 admin admin] PATH_INFO: `/v3/projects/acb9d2f6598045e892d37f6bad5cbee9/users/9031f264047547ac9b589b8f880a4c6c/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:29.814912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0a048-10b1-4729-87f8-978c7c97a953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb9d2f6598045e892d37f6bad5cbee9, user_id=9031f264047547ac9b589b8f880a4c6c, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:29.817576 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0a048-10b1-4729-87f8-978c7c97a953 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:29.846658 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1104/2175] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:29 2022] PUT /identity/v3/projects/acb9d2f6598045e892d37f6bad5cbee9/users/9031f264047547ac9b589b8f880a4c6c/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:29.852491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac157a18-931a-4fee-987c-f42347577359 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:29.852804 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac157a18-931a-4fee-987c-f42347577359 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:29.853057 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac157a18-931a-4fee-987c-f42347577359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:29.890423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ac157a18-931a-4fee-987c-f42347577359 None None] MFA Rules not processed for user `9031f264047547ac9b589b8f880a4c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:29.892655 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ac157a18-931a-4fee-987c-f42347577359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:29.982926 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1072/2176] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:29.988045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0a048-10b1-4729-87f8-978c7c97a953 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:30.079569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0a048-10b1-4729-87f8-978c7c97a953 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:30.085194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92277c4-b21f-4b26-bbf6-0a3c3d1db80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:30.085870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92277c4-b21f-4b26-bbf6-0a3c3d1db80e admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:30.086154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92277c4-b21f-4b26-bbf6-0a3c3d1db80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:30.086503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92277c4-b21f-4b26-bbf6-0a3c3d1db80e admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:30.087213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92277c4-b21f-4b26-bbf6-0a3c3d1db80e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:30.090889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92277c4-b21f-4b26-bbf6-0a3c3d1db80e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:30.123544 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1105/2177] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:29 2022] POST /identity/v3/projects => generated 393 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:30.128053 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac157a18-931a-4fee-987c-f42347577359 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:30.140239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac157a18-931a-4fee-987c-f42347577359 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:30.215836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cb1e8-8aef-417c-8d3a-0d1558774db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:30.217248 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cb1e8-8aef-417c-8d3a-0d1558774db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:30.218166 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cb1e8-8aef-417c-8d3a-0d1558774db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:30.218480 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cb1e8-8aef-417c-8d3a-0d1558774db4 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:30.219329 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cb1e8-8aef-417c-8d3a-0d1558774db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:30.221286 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cb1e8-8aef-417c-8d3a-0d1558774db4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:30.247875 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1073/2178] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:30 2022] POST /identity/v3/users => generated 303 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:30.250438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92277c4-b21f-4b26-bbf6-0a3c3d1db80e admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:30.263304 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92277c4-b21f-4b26-bbf6-0a3c3d1db80e admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:30.269127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65f560-2491-4cb3-8704-f07e43501c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:30.269866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65f560-2491-4cb3-8704-f07e43501c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:30.270213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65f560-2491-4cb3-8704-f07e43501c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:30.270517 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65f560-2491-4cb3-8704-f07e43501c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:30.271243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65f560-2491-4cb3-8704-f07e43501c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:30.272847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65f560-2491-4cb3-8704-f07e43501c59 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:30.284790 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1106/2179] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:30.288987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cb1e8-8aef-417c-8d3a-0d1558774db4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:30.303246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cb1e8-8aef-417c-8d3a-0d1558774db4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:30.308567 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835613f-b1ee-4449-8250-a9cf551bffa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:30.309939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835613f-b1ee-4449-8250-a9cf551bffa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:30.310440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835613f-b1ee-4449-8250-a9cf551bffa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:30.311265 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835613f-b1ee-4449-8250-a9cf551bffa0 admin admin] PATH_INFO: `/v3/projects/8eb26805aab94f448daedd62ea530515/users/e30fcf4dd4eb466b8d52bc429d145837/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:30.315335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:30.330011 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835613f-b1ee-4449-8250-a9cf551bffa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb26805aab94f448daedd62ea530515, user_id=e30fcf4dd4eb466b8d52bc429d145837, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:30.334453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835613f-b1ee-4449-8250-a9cf551bffa0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:30.355789 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1074/2180] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:30 2022] PUT /identity/v3/projects/8eb26805aab94f448daedd62ea530515/users/e30fcf4dd4eb466b8d52bc429d145837/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:30.360038 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65f560-2491-4cb3-8704-f07e43501c59 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:30.371940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65f560-2491-4cb3-8704-f07e43501c59 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:30.463395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:30.487200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8665bc-c8cb-4b6b-aebb-5778eb3058b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:30.488386 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8665bc-c8cb-4b6b-aebb-5778eb3058b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:30.489894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8665bc-c8cb-4b6b-aebb-5778eb3058b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:30.490182 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8665bc-c8cb-4b6b-aebb-5778eb3058b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:30.491061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8665bc-c8cb-4b6b-aebb-5778eb3058b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:30.495916 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8665bc-c8cb-4b6b-aebb-5778eb3058b0 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:30.510896 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1107/2181] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:30.514695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835613f-b1ee-4449-8250-a9cf551bffa0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:30.528451 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835613f-b1ee-4449-8250-a9cf551bffa0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:30.538334 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3147e3-518a-43c3-ba0e-95ec0bd988a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:30.539755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3147e3-518a-43c3-ba0e-95ec0bd988a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:30.541695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3147e3-518a-43c3-ba0e-95ec0bd988a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:30.542272 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3147e3-518a-43c3-ba0e-95ec0bd988a4 admin admin] PATH_INFO: `/v3/projects/8eb26805aab94f448daedd62ea530515/users/e30fcf4dd4eb466b8d52bc429d145837/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:30.545755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3147e3-518a-43c3-ba0e-95ec0bd988a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb26805aab94f448daedd62ea530515, user_id=e30fcf4dd4eb466b8d52bc429d145837, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:30.549817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3147e3-518a-43c3-ba0e-95ec0bd988a4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:30.574717 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1075/2182] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:30 2022] PUT /identity/v3/projects/8eb26805aab94f448daedd62ea530515/users/e30fcf4dd4eb466b8d52bc429d145837/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:30.576213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03543f8-b33f-468c-ad98-8f2526ac064e None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:30.576733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03543f8-b33f-468c-ad98-8f2526ac064e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:30.577078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03543f8-b33f-468c-ad98-8f2526ac064e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:30.631230 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e03543f8-b33f-468c-ad98-8f2526ac064e None None] MFA Rules not processed for user `e30fcf4dd4eb466b8d52bc429d145837`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:30.632922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e03543f8-b33f-468c-ad98-8f2526ac064e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:30.739215 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1108/2183] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:30 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:30.758316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3147e3-518a-43c3-ba0e-95ec0bd988a4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:30.758521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bd3147e3-518a-43c3-ba0e-95ec0bd988a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:30.836391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3147e3-518a-43c3-ba0e-95ec0bd988a4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:30.840753 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc2013-99c7-45d8-979c-3cf3a6b9fc03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:30.842061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc2013-99c7-45d8-979c-3cf3a6b9fc03 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:30.842496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc2013-99c7-45d8-979c-3cf3a6b9fc03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:30.842891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc2013-99c7-45d8-979c-3cf3a6b9fc03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:30.855407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc2013-99c7-45d8-979c-3cf3a6b9fc03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:30.856888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc2013-99c7-45d8-979c-3cf3a6b9fc03 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:30.868545 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1076/2184] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:30.925284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03543f8-b33f-468c-ad98-8f2526ac064e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:30.926907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e03543f8-b33f-468c-ad98-8f2526ac064e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:30.932118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc2013-99c7-45d8-979c-3cf3a6b9fc03 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:30.934720 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-26dc2013-99c7-45d8-979c-3cf3a6b9fc03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:30.953935 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03543f8-b33f-468c-ad98-8f2526ac064e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:30.965770 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6243a-e0a3-4c20-a290-1946e619b0ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:30.966630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6243a-e0a3-4c20-a290-1946e619b0ac service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:30.966754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6243a-e0a3-4c20-a290-1946e619b0ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:30.967009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6243a-e0a3-4c20-a290-1946e619b0ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:31.123010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc2013-99c7-45d8-979c-3cf3a6b9fc03 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:31.125494 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6243a-e0a3-4c20-a290-1946e619b0ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:31.130279 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6243a-e0a3-4c20-a290-1946e619b0ac service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:31.130915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:31.131866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:31.132413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:31.132738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] PATH_INFO: `/v3/projects/0453aba57f6444379d4f2fddcb60d1f7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:31.134428 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0453aba57f6444379d4f2fddcb60d1f7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:31.137859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:31.157386 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1109/2185] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:31.241849 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0453aba57f6444379d4f2fddcb60d1f7', 'request_id': 'req-2f897082-065c-4de4-b22b-77afdde584e9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:31.259605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0453aba57f6444379d4f2fddcb60d1f7', 'request_id': 'req-2f897082-065c-4de4-b22b-77afdde584e9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:31.288521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0453aba57f6444379d4f2fddcb60d1f7', 'request_id': 'req-2f897082-065c-4de4-b22b-77afdde584e9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:31.316271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] The token cache is being invalidate because project 0453aba57f6444379d4f2fddcb60d1f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:31.317047 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:31.317568 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2f897082-065c-4de4-b22b-77afdde584e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:31.324092 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1077/2186] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:30 2022] DELETE /identity/v3/projects/0453aba57f6444379d4f2fddcb60d1f7 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:31.325394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf8a0c-5670-41e1-a13b-48e1ac3f11e2 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:31.325898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf8a0c-5670-41e1-a13b-48e1ac3f11e2 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:31.325989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf8a0c-5670-41e1-a13b-48e1ac3f11e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:31.358583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-8bbf8a0c-5670-41e1-a13b-48e1ac3f11e2 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:31.360307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8bbf8a0c-5670-41e1-a13b-48e1ac3f11e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:31.491578 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1110/2187] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:31.499030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802fd6e-87b9-485d-bd85-6f366ab1fc7b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:31.499320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802fd6e-87b9-485d-bd85-6f366ab1fc7b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:31.499597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802fd6e-87b9-485d-bd85-6f366ab1fc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:31.517935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:31.557437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e802fd6e-87b9-485d-bd85-6f366ab1fc7b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:31.558793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e802fd6e-87b9-485d-bd85-6f366ab1fc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:31.573140 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1078/2188] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:31.574459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf8a0c-5670-41e1-a13b-48e1ac3f11e2 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:31.588321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf8a0c-5670-41e1-a13b-48e1ac3f11e2 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:31.604908 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f217e71-281d-4752-883e-d5147cea7fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:31.605958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f217e71-281d-4752-883e-d5147cea7fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:31.609102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f217e71-281d-4752-883e-d5147cea7fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:31.609418 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f217e71-281d-4752-883e-d5147cea7fb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:31.610168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f217e71-281d-4752-883e-d5147cea7fb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:31.610503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f217e71-281d-4752-883e-d5147cea7fb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:31.615557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f217e71-281d-4752-883e-d5147cea7fb1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:31.625769 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:31.640876 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1111/2189] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:31.645209 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802fd6e-87b9-485d-bd85-6f366ab1fc7b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:31.662363 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802fd6e-87b9-485d-bd85-6f366ab1fc7b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:31.677249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bcf4-42d2-4432-8008-89ea25ee343e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:31.677958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bcf4-42d2-4432-8008-89ea25ee343e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:31.682441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bcf4-42d2-4432-8008-89ea25ee343e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:31.682790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bcf4-42d2-4432-8008-89ea25ee343e admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:31.683577 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692bcf4-42d2-4432-8008-89ea25ee343e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:31.685599 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692bcf4-42d2-4432-8008-89ea25ee343e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:31.740872 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1079/2190] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:31 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:31.745319 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f217e71-281d-4752-883e-d5147cea7fb1 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:31.770621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f217e71-281d-4752-883e-d5147cea7fb1 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:31.928690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f298e15-8a11-4d50-920f-5ea0b7227252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:31.929091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f298e15-8a11-4d50-920f-5ea0b7227252 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:31.929713 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f298e15-8a11-4d50-920f-5ea0b7227252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:31.930203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f298e15-8a11-4d50-920f-5ea0b7227252 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:31.932169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f298e15-8a11-4d50-920f-5ea0b7227252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:31.936387 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f298e15-8a11-4d50-920f-5ea0b7227252 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:31.989586 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1112/2191] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:31 2022] POST /identity/v3/users => generated 285 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:31.994266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bcf4-42d2-4432-8008-89ea25ee343e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:32.009025 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bcf4-42d2-4432-8008-89ea25ee343e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:32.015724 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75e30c-093e-4299-ad7f-5f5942765dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:32.016473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e30c-093e-4299-ad7f-5f5942765dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:32.018102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e30c-093e-4299-ad7f-5f5942765dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:32.018102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e30c-093e-4299-ad7f-5f5942765dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:32.034917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75e30c-093e-4299-ad7f-5f5942765dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:32.039400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75e30c-093e-4299-ad7f-5f5942765dcc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:32.078902 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1080/2192] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:32.083456 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f298e15-8a11-4d50-920f-5ea0b7227252 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:32.105531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f298e15-8a11-4d50-920f-5ea0b7227252 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:32.122055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec7f98-2964-47b7-aa76-710594977ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:32.123565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec7f98-2964-47b7-aa76-710594977ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:32.124617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec7f98-2964-47b7-aa76-710594977ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:32.126682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec7f98-2964-47b7-aa76-710594977ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:32.129641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec7f98-2964-47b7-aa76-710594977ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:32.138442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec7f98-2964-47b7-aa76-710594977ad5 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:32.171014 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1113/2193] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:32.175208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75e30c-093e-4299-ad7f-5f5942765dcc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:32.191103 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75e30c-093e-4299-ad7f-5f5942765dcc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:32.206993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aeb61e-8d98-40af-8959-be50aae81065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:32.207978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aeb61e-8d98-40af-8959-be50aae81065 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:32.208265 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aeb61e-8d98-40af-8959-be50aae81065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:32.208545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aeb61e-8d98-40af-8959-be50aae81065 admin admin] PATH_INFO: `/v3/projects/9e0cf774c7a4462b90c943c89bc07c97/users/dd6cc8cb22bb4adca166abfb731ebd04/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:32.247488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aeb61e-8d98-40af-8959-be50aae81065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0cf774c7a4462b90c943c89bc07c97, user_id=dd6cc8cb22bb4adca166abfb731ebd04, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:32.250174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aeb61e-8d98-40af-8959-be50aae81065 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:32.289480 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1081/2194] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:32 2022] PUT /identity/v3/projects/9e0cf774c7a4462b90c943c89bc07c97/users/dd6cc8cb22bb4adca166abfb731ebd04/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:32.387406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e78e9a-15b1-4494-a6de-42b5166edc75 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:32.387406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e78e9a-15b1-4494-a6de-42b5166edc75 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:32.388024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e78e9a-15b1-4494-a6de-42b5166edc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:32.437384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e1e78e9a-15b1-4494-a6de-42b5166edc75 None None] MFA Rules not processed for user `dd6cc8cb22bb4adca166abfb731ebd04`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:32.439062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e1e78e9a-15b1-4494-a6de-42b5166edc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:32.659449 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1114/2195] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:32.670393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aeb61e-8d98-40af-8959-be50aae81065 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:32.672640 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-70aeb61e-8d98-40af-8959-be50aae81065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:32.774651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aeb61e-8d98-40af-8959-be50aae81065 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:32.779443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babff9b-3573-43a5-a5ec-d5ee785cb431 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b97736e49e2a4bfb9cf7093cfa8b2b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:32.779813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babff9b-3573-43a5-a5ec-d5ee785cb431 service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:32.780093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babff9b-3573-43a5-a5ec-d5ee785cb431 service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:32.780365 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babff9b-3573-43a5-a5ec-d5ee785cb431 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:32.792554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babff9b-3573-43a5-a5ec-d5ee785cb431 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:32.794150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babff9b-3573-43a5-a5ec-d5ee785cb431 service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:32.808697 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1082/2196] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:34.310505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e78e9a-15b1-4494-a6de-42b5166edc75 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:34.311960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e1e78e9a-15b1-4494-a6de-42b5166edc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:34.401382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e78e9a-15b1-4494-a6de-42b5166edc75 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:34.405141 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d103214-b7f2-47dc-b58d-184d837f3e2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:34.405894 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d103214-b7f2-47dc-b58d-184d837f3e2b service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:34.406289 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d103214-b7f2-47dc-b58d-184d837f3e2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:34.406566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d103214-b7f2-47dc-b58d-184d837f3e2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:34.408369 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5d103214-b7f2-47dc-b58d-184d837f3e2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:34.429666 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:34.476546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d103214-b7f2-47dc-b58d-184d837f3e2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:34.478018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d103214-b7f2-47dc-b58d-184d837f3e2b service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:34.488018 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1115/2197] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:35.761771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babff9b-3573-43a5-a5ec-d5ee785cb431 service glance] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:35.766298 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2babff9b-3573-43a5-a5ec-d5ee785cb431 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:35.837446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babff9b-3573-43a5-a5ec-d5ee785cb431 service glance] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:35.842661 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:35.843552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:35.843843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:35.844116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] PATH_INFO: `/v3/users/9031f264047547ac9b589b8f880a4c6c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:35.845315 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9031f264047547ac9b589b8f880a4c6c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:35.847307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:35.895021 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:35.896671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9031f264047547ac9b589b8f880a4c6c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:35.897467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9031f264047547ac9b589b8f880a4c6c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:35.898165 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9031f264047547ac9b589b8f880a4c6c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:35.908481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9031f264047547ac9b589b8f880a4c6c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:35.920134 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9031f264047547ac9b589b8f880a4c6c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:35.934339 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1083/2198] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:35 2022] DELETE /identity/v3/users/9031f264047547ac9b589b8f880a4c6c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:35.938979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4df9e-3d3b-4e64-94f5-2b9d3a993721 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:35.939522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4df9e-3d3b-4e64-94f5-2b9d3a993721 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:35.939522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4df9e-3d3b-4e64-94f5-2b9d3a993721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:35.973855 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-cdc4df9e-3d3b-4e64-94f5-2b9d3a993721 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:35.975136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-cdc4df9e-3d3b-4e64-94f5-2b9d3a993721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:36.038480 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1116/2199] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:36.048529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:36.049970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:36.107447 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f56af-7f3d-4938-98f7-68d583de3954 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:36.111080 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec11deb-2bd1-458d-886d-c57bf78604ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:36.111731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec11deb-2bd1-458d-886d-c57bf78604ce service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:36.112012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec11deb-2bd1-458d-886d-c57bf78604ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:36.112293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec11deb-2bd1-458d-886d-c57bf78604ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:36.122509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec11deb-2bd1-458d-886d-c57bf78604ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:36.124813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec11deb-2bd1-458d-886d-c57bf78604ce service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:36.136007 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1084/2200] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:36.433574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4df9e-3d3b-4e64-94f5-2b9d3a993721 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:36.439901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-cdc4df9e-3d3b-4e64-94f5-2b9d3a993721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:36.472483 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4df9e-3d3b-4e64-94f5-2b9d3a993721 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:36.477466 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:36.478665 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:36.478949 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:36.479227 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] PATH_INFO: `/v3/projects/acb9d2f6598045e892d37f6bad5cbee9` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:36.480606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb9d2f6598045e892d37f6bad5cbee9)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:36.482517 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:36.544890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb9d2f6598045e892d37f6bad5cbee9', 'request_id': 'req-f1e2e782-3b7e-4edd-8987-970aca377fb7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:36.551650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb9d2f6598045e892d37f6bad5cbee9', 'request_id': 'req-f1e2e782-3b7e-4edd-8987-970aca377fb7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:36.560954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb9d2f6598045e892d37f6bad5cbee9', 'request_id': 'req-f1e2e782-3b7e-4edd-8987-970aca377fb7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:36.574124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] The token cache is being invalidate because project acb9d2f6598045e892d37f6bad5cbee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:36.574426 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:36.575032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:36.576791 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1117/2201] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:36 2022] DELETE /identity/v3/projects/acb9d2f6598045e892d37f6bad5cbee9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:36.580712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec11deb-2bd1-458d-886d-c57bf78604ce service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:36.592537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec11deb-2bd1-458d-886d-c57bf78604ce service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:36.670532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fc003-fc7d-44ba-83cf-c38232f38795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:36.671923 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fc003-fc7d-44ba-83cf-c38232f38795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:36.672207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fc003-fc7d-44ba-83cf-c38232f38795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:36.672485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fc003-fc7d-44ba-83cf-c38232f38795 admin admin] PATH_INFO: `/v3/users/e30fcf4dd4eb466b8d52bc429d145837` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:36.673967 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fc003-fc7d-44ba-83cf-c38232f38795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e30fcf4dd4eb466b8d52bc429d145837)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:36.675684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fc003-fc7d-44ba-83cf-c38232f38795 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:36.734244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-717fc003-fc7d-44ba-83cf-c38232f38795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e30fcf4dd4eb466b8d52bc429d145837'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:36.734890 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-717fc003-fc7d-44ba-83cf-c38232f38795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e30fcf4dd4eb466b8d52bc429d145837'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:36.735502 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-717fc003-fc7d-44ba-83cf-c38232f38795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e30fcf4dd4eb466b8d52bc429d145837'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:36.742841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-717fc003-fc7d-44ba-83cf-c38232f38795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e30fcf4dd4eb466b8d52bc429d145837'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:36.753255 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:36.776637 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-717fc003-fc7d-44ba-83cf-c38232f38795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e30fcf4dd4eb466b8d52bc429d145837'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:36.801827 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1085/2202] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:36 2022] DELETE /identity/v3/users/e30fcf4dd4eb466b8d52bc429d145837 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:37.121342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:37.122899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:37.136203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2e782-3b7e-4edd-8987-970aca377fb7 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:37.185984 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:37.206424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:37.207498 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:37.207786 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:37.208069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] PATH_INFO: `/v3/projects/8eb26805aab94f448daedd62ea530515` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:37.209690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb26805aab94f448daedd62ea530515)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:37.211686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:37.261271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb26805aab94f448daedd62ea530515', 'request_id': 'req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:37.267467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb26805aab94f448daedd62ea530515', 'request_id': 'req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:37.273990 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb26805aab94f448daedd62ea530515', 'request_id': 'req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:37.292440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] The token cache is being invalidate because project 8eb26805aab94f448daedd62ea530515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:37.292752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:37.293432 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e6f4517f-bcda-487d-9ff0-3f34cd5de21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:37.295703 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1118/2203] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:37 2022] DELETE /identity/v3/projects/8eb26805aab94f448daedd62ea530515 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:37.303716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32f1e6-b93e-4daf-8400-01bf4bb85b31 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:37.304159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32f1e6-b93e-4daf-8400-01bf4bb85b31 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:37.304573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32f1e6-b93e-4daf-8400-01bf4bb85b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:37.330561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-cb32f1e6-b93e-4daf-8400-01bf4bb85b31 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:37.332315 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cb32f1e6-b93e-4daf-8400-01bf4bb85b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:37.408684 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1086/2204] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:37.417572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c9eba-7d6e-433e-8be4-dd542a9d5ab8 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:37.417978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c9eba-7d6e-433e-8be4-dd542a9d5ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:37.418389 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c9eba-7d6e-433e-8be4-dd542a9d5ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:37.440081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-574c9eba-7d6e-433e-8be4-dd542a9d5ab8 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:37.441423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-574c9eba-7d6e-433e-8be4-dd542a9d5ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:37.450258 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1119/2205] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:37.454592 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32f1e6-b93e-4daf-8400-01bf4bb85b31 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:37.465159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32f1e6-b93e-4daf-8400-01bf4bb85b31 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:37.470263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b53e6-c456-4c4f-b306-940fd3df2407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:37.470759 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b53e6-c456-4c4f-b306-940fd3df2407 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:37.471136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b53e6-c456-4c4f-b306-940fd3df2407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:37.471481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b53e6-c456-4c4f-b306-940fd3df2407 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:37.472295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b53e6-c456-4c4f-b306-940fd3df2407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:37.472682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b53e6-c456-4c4f-b306-940fd3df2407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:37.474459 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b53e6-c456-4c4f-b306-940fd3df2407 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:37.490259 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1087/2206] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:37.494425 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c9eba-7d6e-433e-8be4-dd542a9d5ab8 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:37.504793 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c9eba-7d6e-433e-8be4-dd542a9d5ab8 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:37.510754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d67e6f-9c7f-4e3a-a62f-46a74ae89730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:37.511643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d67e6f-9c7f-4e3a-a62f-46a74ae89730 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:37.511872 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d67e6f-9c7f-4e3a-a62f-46a74ae89730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:37.512151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d67e6f-9c7f-4e3a-a62f-46a74ae89730 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:37.512851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d67e6f-9c7f-4e3a-a62f-46a74ae89730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:37.514831 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d67e6f-9c7f-4e3a-a62f-46a74ae89730 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:37.546665 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1120/2207] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:37 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:37.551803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b53e6-c456-4c4f-b306-940fd3df2407 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:37.563287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b53e6-c456-4c4f-b306-940fd3df2407 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:37.646208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ed649-e005-4c78-a81d-32554dc08970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:37.647116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed649-e005-4c78-a81d-32554dc08970 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:37.647547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed649-e005-4c78-a81d-32554dc08970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:37.647827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed649-e005-4c78-a81d-32554dc08970 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:37.648531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ed649-e005-4c78-a81d-32554dc08970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:37.650603 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ed649-e005-4c78-a81d-32554dc08970 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:37.681816 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1088/2208] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:37 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:37.686043 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d67e6f-9c7f-4e3a-a62f-46a74ae89730 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:37.703336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d67e6f-9c7f-4e3a-a62f-46a74ae89730 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:37.719290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a5edd-23e2-4be1-8dc9-3a252d4980cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:37.720305 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a5edd-23e2-4be1-8dc9-3a252d4980cd admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:37.720798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a5edd-23e2-4be1-8dc9-3a252d4980cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:37.721266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a5edd-23e2-4be1-8dc9-3a252d4980cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:37.722246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a5edd-23e2-4be1-8dc9-3a252d4980cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:37.724850 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a5edd-23e2-4be1-8dc9-3a252d4980cd admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:37.751825 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1121/2209] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:37.753201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ed649-e005-4c78-a81d-32554dc08970 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:37.766264 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ed649-e005-4c78-a81d-32554dc08970 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:37.776252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d20e95-8c72-4abb-9a02-d6058b41b12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:37.777524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20e95-8c72-4abb-9a02-d6058b41b12c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:37.778535 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20e95-8c72-4abb-9a02-d6058b41b12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:37.779013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20e95-8c72-4abb-9a02-d6058b41b12c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:37.779980 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d20e95-8c72-4abb-9a02-d6058b41b12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:37.782098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d20e95-8c72-4abb-9a02-d6058b41b12c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:37.796092 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1089/2210] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:37.800621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a5edd-23e2-4be1-8dc9-3a252d4980cd admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:37.813290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a5edd-23e2-4be1-8dc9-3a252d4980cd admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:37.818628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbcf77-e269-4cfa-a5cd-b3c1e02c458d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:37.819782 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbcf77-e269-4cfa-a5cd-b3c1e02c458d admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:37.820373 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbcf77-e269-4cfa-a5cd-b3c1e02c458d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:37.821099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbcf77-e269-4cfa-a5cd-b3c1e02c458d admin admin] PATH_INFO: `/v3/projects/0d63971c6cea4b5db874259b474d983e/users/68f269d9b70d49a9b3d8a5fa1a188f4f/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:37.846615 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbcf77-e269-4cfa-a5cd-b3c1e02c458d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d63971c6cea4b5db874259b474d983e, user_id=68f269d9b70d49a9b3d8a5fa1a188f4f, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:37.849733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbcf77-e269-4cfa-a5cd-b3c1e02c458d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:37.883500 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1122/2211] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:37 2022] PUT /identity/v3/projects/0d63971c6cea4b5db874259b474d983e/users/68f269d9b70d49a9b3d8a5fa1a188f4f/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:37.887967 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d80fec-c860-4547-bd40-74361aa04879 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:37.888544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d80fec-c860-4547-bd40-74361aa04879 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:37.889030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d80fec-c860-4547-bd40-74361aa04879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:37.965055 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e4d80fec-c860-4547-bd40-74361aa04879 None None] MFA Rules not processed for user `68f269d9b70d49a9b3d8a5fa1a188f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:37.974150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e4d80fec-c860-4547-bd40-74361aa04879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:38.081943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:38.148925 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1090/2212] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:38.153670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbcf77-e269-4cfa-a5cd-b3c1e02c458d admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:38.274308 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbcf77-e269-4cfa-a5cd-b3c1e02c458d admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:38.280286 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32360903-0ad3-43fc-aa08-0cec8d5cdce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:38.281276 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32360903-0ad3-43fc-aa08-0cec8d5cdce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:38.281826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32360903-0ad3-43fc-aa08-0cec8d5cdce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:38.282477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32360903-0ad3-43fc-aa08-0cec8d5cdce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:38.283387 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32360903-0ad3-43fc-aa08-0cec8d5cdce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:38.285834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32360903-0ad3-43fc-aa08-0cec8d5cdce0 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:38.299868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:38.326433 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1123/2213] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:38 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:38.331539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d80fec-c860-4547-bd40-74361aa04879 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:38.345096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d80fec-c860-4547-bd40-74361aa04879 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:38.413312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98911db2-caef-4040-bf74-10260b7c544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:38.414415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911db2-caef-4040-bf74-10260b7c544c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:38.414990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911db2-caef-4040-bf74-10260b7c544c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:38.415433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911db2-caef-4040-bf74-10260b7c544c admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:38.416626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98911db2-caef-4040-bf74-10260b7c544c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:38.418822 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98911db2-caef-4040-bf74-10260b7c544c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:38.444384 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1091/2214] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:38 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:38.448365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32360903-0ad3-43fc-aa08-0cec8d5cdce0 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:38.459459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32360903-0ad3-43fc-aa08-0cec8d5cdce0 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:38.464987 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3c5c9-4cb4-4876-9a96-f2ab139b810e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:38.465710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3c5c9-4cb4-4876-9a96-f2ab139b810e admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:38.465999 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3c5c9-4cb4-4876-9a96-f2ab139b810e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:38.466473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3c5c9-4cb4-4876-9a96-f2ab139b810e admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:38.468109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3c5c9-4cb4-4876-9a96-f2ab139b810e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:38.469873 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3c5c9-4cb4-4876-9a96-f2ab139b810e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:38.489324 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1124/2215] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:38.493100 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98911db2-caef-4040-bf74-10260b7c544c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:38.505843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98911db2-caef-4040-bf74-10260b7c544c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:38.515673 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d18f9-2e5e-478d-ac9e-c2900ec36ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:38.517042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d18f9-2e5e-478d-ac9e-c2900ec36ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:38.517180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d18f9-2e5e-478d-ac9e-c2900ec36ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:38.517454 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d18f9-2e5e-478d-ac9e-c2900ec36ecb admin admin] PATH_INFO: `/v3/projects/72727482fc914018b76fb68789eb4b03/users/d8bc6a064ed3421e8eefd11423a6fe82/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:38.542912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d18f9-2e5e-478d-ac9e-c2900ec36ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=72727482fc914018b76fb68789eb4b03, user_id=d8bc6a064ed3421e8eefd11423a6fe82, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:38.547665 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d18f9-2e5e-478d-ac9e-c2900ec36ecb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:38.562014 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1092/2216] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:38 2022] PUT /identity/v3/projects/72727482fc914018b76fb68789eb4b03/users/d8bc6a064ed3421e8eefd11423a6fe82/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:38.565891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3c5c9-4cb4-4876-9a96-f2ab139b810e admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:38.577606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3c5c9-4cb4-4876-9a96-f2ab139b810e admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:38.679561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e1d8b-b278-47bb-80f4-5b9a7f004d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:38.680574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e1d8b-b278-47bb-80f4-5b9a7f004d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:38.680896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e1d8b-b278-47bb-80f4-5b9a7f004d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:38.681169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e1d8b-b278-47bb-80f4-5b9a7f004d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:38.682029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e1d8b-b278-47bb-80f4-5b9a7f004d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:38.686279 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e1d8b-b278-47bb-80f4-5b9a7f004d4c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:38.705091 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1125/2217] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:38.706375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d18f9-2e5e-478d-ac9e-c2900ec36ecb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:38.720618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d18f9-2e5e-478d-ac9e-c2900ec36ecb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:38.722948 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb576b-0080-4261-bab6-16fd1bdcafdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:38.723943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb576b-0080-4261-bab6-16fd1bdcafdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:38.724316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb576b-0080-4261-bab6-16fd1bdcafdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:38.724699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb576b-0080-4261-bab6-16fd1bdcafdb admin admin] PATH_INFO: `/v3/projects/72727482fc914018b76fb68789eb4b03/users/d8bc6a064ed3421e8eefd11423a6fe82/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:38.726842 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb576b-0080-4261-bab6-16fd1bdcafdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=72727482fc914018b76fb68789eb4b03, user_id=d8bc6a064ed3421e8eefd11423a6fe82, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:38.729634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb576b-0080-4261-bab6-16fd1bdcafdb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:38.740637 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1093/2218] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:38 2022] PUT /identity/v3/projects/72727482fc914018b76fb68789eb4b03/users/d8bc6a064ed3421e8eefd11423a6fe82/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:38.751038 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193ae97-b058-4398-822f-052c16b29f7c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:38.751329 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193ae97-b058-4398-822f-052c16b29f7c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:38.751609 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193ae97-b058-4398-822f-052c16b29f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:38.787801 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-0193ae97-b058-4398-822f-052c16b29f7c None None] MFA Rules not processed for user `d8bc6a064ed3421e8eefd11423a6fe82`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:38.789091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0193ae97-b058-4398-822f-052c16b29f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:38.869981 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1126/2219] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:38.885634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb576b-0080-4261-bab6-16fd1bdcafdb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:38.896034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-39fb576b-0080-4261-bab6-16fd1bdcafdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:39.015571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb576b-0080-4261-bab6-16fd1bdcafdb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:39.023133 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f608f8-f492-4c60-acd1-f28dba104855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:39.024286 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f608f8-f492-4c60-acd1-f28dba104855 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:39.024875 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f608f8-f492-4c60-acd1-f28dba104855 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:39.025424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f608f8-f492-4c60-acd1-f28dba104855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:39.038188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f608f8-f492-4c60-acd1-f28dba104855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:39.040511 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f608f8-f492-4c60-acd1-f28dba104855 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:39.069535 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1094/2220] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:39.259669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193ae97-b058-4398-822f-052c16b29f7c None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:39.291607 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193ae97-b058-4398-822f-052c16b29f7c None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:39.307386 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04be47e-52d0-4d43-8c06-319cd577a9a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:39.308448 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04be47e-52d0-4d43-8c06-319cd577a9a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:39.308926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04be47e-52d0-4d43-8c06-319cd577a9a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:39.309401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04be47e-52d0-4d43-8c06-319cd577a9a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:39.411002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04be47e-52d0-4d43-8c06-319cd577a9a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:39.412719 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04be47e-52d0-4d43-8c06-319cd577a9a6 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:39.417861 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:39.426762 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1127/2221] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:40.738726 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f608f8-f492-4c60-acd1-f28dba104855 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:40.742634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-81f608f8-f492-4c60-acd1-f28dba104855 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:40.830313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:40.878434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f608f8-f492-4c60-acd1-f28dba104855 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:40.883632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472007a-a06f-4af5-8d7c-403666cccc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:40.885821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472007a-a06f-4af5-8d7c-403666cccc53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:40.886100 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472007a-a06f-4af5-8d7c-403666cccc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:40.886545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472007a-a06f-4af5-8d7c-403666cccc53 admin admin] PATH_INFO: `/v3/users/7d4bb910a5bc4c7ea2667dc65066b2bd` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:40.887943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472007a-a06f-4af5-8d7c-403666cccc53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4bb910a5bc4c7ea2667dc65066b2bd)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:40.889955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472007a-a06f-4af5-8d7c-403666cccc53 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:40.989756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6472007a-a06f-4af5-8d7c-403666cccc53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4bb910a5bc4c7ea2667dc65066b2bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:40.990584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6472007a-a06f-4af5-8d7c-403666cccc53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4bb910a5bc4c7ea2667dc65066b2bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:40.991233 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6472007a-a06f-4af5-8d7c-403666cccc53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4bb910a5bc4c7ea2667dc65066b2bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:40.999606 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6472007a-a06f-4af5-8d7c-403666cccc53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4bb910a5bc4c7ea2667dc65066b2bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.013058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6472007a-a06f-4af5-8d7c-403666cccc53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4bb910a5bc4c7ea2667dc65066b2bd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.040719 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1095/2222] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:40 2022] DELETE /identity/v3/users/7d4bb910a5bc4c7ea2667dc65066b2bd => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:41.233898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04be47e-52d0-4d43-8c06-319cd577a9a6 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:41.247044 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04be47e-52d0-4d43-8c06-319cd577a9a6 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:41.315233 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:41.316212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.317994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.317994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] PATH_INFO: `/v3/projects/cf8cdaba37244ff9bcf6df35373f1467` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.318815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf8cdaba37244ff9bcf6df35373f1467)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:41.320801 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:41.394975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf8cdaba37244ff9bcf6df35373f1467', 'request_id': 'req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.399821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf8cdaba37244ff9bcf6df35373f1467', 'request_id': 'req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.414588 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf8cdaba37244ff9bcf6df35373f1467', 'request_id': 'req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.432009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] The token cache is being invalidate because project cf8cdaba37244ff9bcf6df35373f1467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:41.432299 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.432954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ba458bc6-a92a-4e77-9fd0-6c9b91a6de71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.434828 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1128/2223] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:41 2022] DELETE /identity/v3/projects/cf8cdaba37244ff9bcf6df35373f1467 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:41.440301 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecfd02-aabf-4518-b4f3-f15cd1e0ba79 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.440621 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecfd02-aabf-4518-b4f3-f15cd1e0ba79 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.440955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecfd02-aabf-4518-b4f3-f15cd1e0ba79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.462556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0eecfd02-aabf-4518-b4f3-f15cd1e0ba79 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:41.463794 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0eecfd02-aabf-4518-b4f3-f15cd1e0ba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:41.530644 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1096/2224] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:41.542833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c7aec-a6cc-4882-9d07-80d338e457a7 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.543121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c7aec-a6cc-4882-9d07-80d338e457a7 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.543398 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c7aec-a6cc-4882-9d07-80d338e457a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.563000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-be2c7aec-a6cc-4882-9d07-80d338e457a7 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:41.564177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-be2c7aec-a6cc-4882-9d07-80d338e457a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:41.573030 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1129/2225] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:41.577817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecfd02-aabf-4518-b4f3-f15cd1e0ba79 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:41.587978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecfd02-aabf-4518-b4f3-f15cd1e0ba79 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:41.592588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164629a8-a553-45ff-bb84-a96e47ddaee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:41.593239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164629a8-a553-45ff-bb84-a96e47ddaee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.593520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164629a8-a553-45ff-bb84-a96e47ddaee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.593794 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164629a8-a553-45ff-bb84-a96e47ddaee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.594554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164629a8-a553-45ff-bb84-a96e47ddaee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:41.594859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164629a8-a553-45ff-bb84-a96e47ddaee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:41.596452 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164629a8-a553-45ff-bb84-a96e47ddaee3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:41.608790 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1097/2226] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:41.613189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c7aec-a6cc-4882-9d07-80d338e457a7 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:41.622537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c7aec-a6cc-4882-9d07-80d338e457a7 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:41.627407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e4265-b99d-4301-803b-e66f75a29789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:41.628092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e4265-b99d-4301-803b-e66f75a29789 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.628370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e4265-b99d-4301-803b-e66f75a29789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.628648 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e4265-b99d-4301-803b-e66f75a29789 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.629330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e4265-b99d-4301-803b-e66f75a29789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:41.631271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e4265-b99d-4301-803b-e66f75a29789 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:41.659312 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1130/2227] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:41 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:41.663536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164629a8-a553-45ff-bb84-a96e47ddaee3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:41.674303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164629a8-a553-45ff-bb84-a96e47ddaee3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:41.717567 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:41.736974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c253e0-f40a-4085-80c3-3e82300309e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:41.737973 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c253e0-f40a-4085-80c3-3e82300309e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.738086 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c253e0-f40a-4085-80c3-3e82300309e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.738340 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c253e0-f40a-4085-80c3-3e82300309e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.739037 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c253e0-f40a-4085-80c3-3e82300309e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:41.740981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c253e0-f40a-4085-80c3-3e82300309e6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:41.773720 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1098/2228] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:41 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:41.781948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e4265-b99d-4301-803b-e66f75a29789 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:41.792226 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e4265-b99d-4301-803b-e66f75a29789 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:41.796952 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235d65-1369-4f17-bab0-256590038be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:41.797732 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235d65-1369-4f17-bab0-256590038be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.798110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235d65-1369-4f17-bab0-256590038be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.798500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70235d65-1369-4f17-bab0-256590038be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.800856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70235d65-1369-4f17-bab0-256590038be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:41.801180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70235d65-1369-4f17-bab0-256590038be4 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:41.804755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c253e0-f40a-4085-80c3-3e82300309e6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:41.806677 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-06c253e0-f40a-4085-80c3-3e82300309e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:41.819986 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1131/2229] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:41.821946 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c253e0-f40a-4085-80c3-3e82300309e6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:41.823805 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235d65-1369-4f17-bab0-256590038be4 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:41.828017 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c495792-8f19-4b97-a1d2-514d1640d5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:41.829089 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c495792-8f19-4b97-a1d2-514d1640d5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.829378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c495792-8f19-4b97-a1d2-514d1640d5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.829659 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c495792-8f19-4b97-a1d2-514d1640d5c8 admin admin] PATH_INFO: `/v3/users/454a5322c0b84e9190da70b3bd5d8f22` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.831590 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c495792-8f19-4b97-a1d2-514d1640d5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=454a5322c0b84e9190da70b3bd5d8f22)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:41.834039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c495792-8f19-4b97-a1d2-514d1640d5c8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:41.834173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70235d65-1369-4f17-bab0-256590038be4 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:41.839409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363f0f2-e0bd-4de9-9363-9d37c206133f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:41.840150 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363f0f2-e0bd-4de9-9363-9d37c206133f admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.843067 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363f0f2-e0bd-4de9-9363-9d37c206133f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.843067 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363f0f2-e0bd-4de9-9363-9d37c206133f admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.843067 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363f0f2-e0bd-4de9-9363-9d37c206133f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:41.843458 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363f0f2-e0bd-4de9-9363-9d37c206133f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:41.854922 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1132/2230] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:41.860191 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363f0f2-e0bd-4de9-9363-9d37c206133f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:41.878803 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363f0f2-e0bd-4de9-9363-9d37c206133f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:41.885754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e66928-09c4-41cc-98b1-27cb9b3a5cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:41.886889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e66928-09c4-41cc-98b1-27cb9b3a5cd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.887149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e66928-09c4-41cc-98b1-27cb9b3a5cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.887425 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e66928-09c4-41cc-98b1-27cb9b3a5cd4 admin admin] PATH_INFO: `/v3/projects/96cc7b005126403f94110029f4242a9f/users/ae9cd6d6d5824aa1932d3353578285af/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.905147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1c495792-8f19-4b97-a1d2-514d1640d5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454a5322c0b84e9190da70b3bd5d8f22'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.906483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1c495792-8f19-4b97-a1d2-514d1640d5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454a5322c0b84e9190da70b3bd5d8f22'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.907097 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1c495792-8f19-4b97-a1d2-514d1640d5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454a5322c0b84e9190da70b3bd5d8f22'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.911027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e66928-09c4-41cc-98b1-27cb9b3a5cd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96cc7b005126403f94110029f4242a9f, user_id=ae9cd6d6d5824aa1932d3353578285af, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:41.913848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e66928-09c4-41cc-98b1-27cb9b3a5cd4 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:41.914340 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1c495792-8f19-4b97-a1d2-514d1640d5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454a5322c0b84e9190da70b3bd5d8f22'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.930915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1c495792-8f19-4b97-a1d2-514d1640d5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454a5322c0b84e9190da70b3bd5d8f22'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:41.934788 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1133/2231] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:41 2022] PUT /identity/v3/projects/96cc7b005126403f94110029f4242a9f/users/ae9cd6d6d5824aa1932d3353578285af/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:41.940219 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87d101-d9f6-45ce-9ec3-48626351372a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.940638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87d101-d9f6-45ce-9ec3-48626351372a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.940928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87d101-d9f6-45ce-9ec3-48626351372a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.952664 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1099/2232] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:41 2022] DELETE /identity/v3/users/454a5322c0b84e9190da70b3bd5d8f22 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:41.957228 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b9536-081a-4de0-8c5b-f9125bdf16e5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:41.957228 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b9536-081a-4de0-8c5b-f9125bdf16e5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:41.957810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b9536-081a-4de0-8c5b-f9125bdf16e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:41.960983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ef87d101-d9f6-45ce-9ec3-48626351372a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:41.962464 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ef87d101-d9f6-45ce-9ec3-48626351372a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:41.973411 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:41.988467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e30b9536-081a-4de0-8c5b-f9125bdf16e5 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:41.994372 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e30b9536-081a-4de0-8c5b-f9125bdf16e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:42.063254 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1100/2233] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:42.075358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b9536-081a-4de0-8c5b-f9125bdf16e5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:42.076740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e30b9536-081a-4de0-8c5b-f9125bdf16e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:42.082355 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1134/2234] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:42.093523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87d101-d9f6-45ce-9ec3-48626351372a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:42.098707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ef87d101-d9f6-45ce-9ec3-48626351372a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:42.136677 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:42.151958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b9536-081a-4de0-8c5b-f9125bdf16e5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:42.158933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8982cc-c18b-4537-8dbe-10e036281a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:42.160484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8982cc-c18b-4537-8dbe-10e036281a38 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:42.160861 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8982cc-c18b-4537-8dbe-10e036281a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:42.161215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8982cc-c18b-4537-8dbe-10e036281a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:42.163046 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87d101-d9f6-45ce-9ec3-48626351372a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:42.167194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036df64a-c690-4336-a9ef-a8a1d8e3f53b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:42.167982 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036df64a-c690-4336-a9ef-a8a1d8e3f53b service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:42.168206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036df64a-c690-4336-a9ef-a8a1d8e3f53b service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:42.171048 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036df64a-c690-4336-a9ef-a8a1d8e3f53b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:42.186378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8982cc-c18b-4537-8dbe-10e036281a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:42.187406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036df64a-c690-4336-a9ef-a8a1d8e3f53b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:42.188237 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8982cc-c18b-4537-8dbe-10e036281a38 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:42.189442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036df64a-c690-4336-a9ef-a8a1d8e3f53b service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:42.204547 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1135/2235] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:42.206530 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1101/2236] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:42.208671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036df64a-c690-4336-a9ef-a8a1d8e3f53b service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:42.210317 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-036df64a-c690-4336-a9ef-a8a1d8e3f53b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:42.227164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036df64a-c690-4336-a9ef-a8a1d8e3f53b service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:42.233995 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:42.235125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:42.235502 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:42.235915 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] PATH_INFO: `/v3/users/68f269d9b70d49a9b3d8a5fa1a188f4f` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:42.237508 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] RBAC: Authorizing `identity:delete_user(user_id=68f269d9b70d49a9b3d8a5fa1a188f4f)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:42.240155 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:42.324726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68f269d9b70d49a9b3d8a5fa1a188f4f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:42.325504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68f269d9b70d49a9b3d8a5fa1a188f4f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:42.350365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68f269d9b70d49a9b3d8a5fa1a188f4f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:42.361082 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68f269d9b70d49a9b3d8a5fa1a188f4f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:42.362592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68f269d9b70d49a9b3d8a5fa1a188f4f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:42.396874 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1136/2237] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:42 2022] DELETE /identity/v3/users/68f269d9b70d49a9b3d8a5fa1a188f4f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:42.402513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b478c-7ffd-4c41-9bfb-3905034238f9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:42.403059 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b478c-7ffd-4c41-9bfb-3905034238f9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:42.403539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b478c-7ffd-4c41-9bfb-3905034238f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:42.443866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5f2b478c-7ffd-4c41-9bfb-3905034238f9 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:42.446376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5f2b478c-7ffd-4c41-9bfb-3905034238f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:42.535654 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1102/2238] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:42.549043 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:42.551065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:42.598939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:42.632180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4980f76-cd8c-4b86-9383-f776f8cf087d admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:42.636190 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee122d1-567b-4110-84b4-4a136a576274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:42.636932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee122d1-567b-4110-84b4-4a136a576274 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:42.637169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee122d1-567b-4110-84b4-4a136a576274 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:42.637435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee122d1-567b-4110-84b4-4a136a576274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:42.654603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee122d1-567b-4110-84b4-4a136a576274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:42.656203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee122d1-567b-4110-84b4-4a136a576274 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:42.668955 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1137/2239] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:42.673612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b478c-7ffd-4c41-9bfb-3905034238f9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:42.675110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5f2b478c-7ffd-4c41-9bfb-3905034238f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:42.707749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b478c-7ffd-4c41-9bfb-3905034238f9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:42.713496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:42.714580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:42.714867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:42.715152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] PATH_INFO: `/v3/projects/2fa3fef1dd9e428e9e13c686a84836f9` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:42.716365 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa3fef1dd9e428e9e13c686a84836f9)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:42.718467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:42.784698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa3fef1dd9e428e9e13c686a84836f9', 'request_id': 'req-0e91a892-8cf8-4826-bade-4d4fe9e52b60'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:42.793107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa3fef1dd9e428e9e13c686a84836f9', 'request_id': 'req-0e91a892-8cf8-4826-bade-4d4fe9e52b60'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:42.799643 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa3fef1dd9e428e9e13c686a84836f9', 'request_id': 'req-0e91a892-8cf8-4826-bade-4d4fe9e52b60'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:42.811318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] The token cache is being invalidate because project 2fa3fef1dd9e428e9e13c686a84836f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:42.811610 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:42.812310 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0e91a892-8cf8-4826-bade-4d4fe9e52b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:42.813916 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1103/2240] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:42 2022] DELETE /identity/v3/projects/2fa3fef1dd9e428e9e13c686a84836f9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:42.819338 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bf284-9c3b-4a45-989e-1721fda2844c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:42.819630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bf284-9c3b-4a45-989e-1721fda2844c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:42.819912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bf284-9c3b-4a45-989e-1721fda2844c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:42.839103 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-782bf284-9c3b-4a45-989e-1721fda2844c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:42.840669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-782bf284-9c3b-4a45-989e-1721fda2844c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:42.912786 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1138/2241] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:42.921708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168ce72-bced-4c16-86b1-13e696a53a5c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:42.921992 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168ce72-bced-4c16-86b1-13e696a53a5c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:42.922267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168ce72-bced-4c16-86b1-13e696a53a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:42.942931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-d168ce72-bced-4c16-86b1-13e696a53a5c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:42.944224 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d168ce72-bced-4c16-86b1-13e696a53a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:42.953390 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1104/2242] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:42.957995 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bf284-9c3b-4a45-989e-1721fda2844c None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:42.968638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bf284-9c3b-4a45-989e-1721fda2844c None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:42.973579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245605ea-af8f-487a-9362-ec3605ab2cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:42.974294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245605ea-af8f-487a-9362-ec3605ab2cce admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:42.974583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245605ea-af8f-487a-9362-ec3605ab2cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:42.974862 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245605ea-af8f-487a-9362-ec3605ab2cce admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:42.975628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245605ea-af8f-487a-9362-ec3605ab2cce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:42.975943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245605ea-af8f-487a-9362-ec3605ab2cce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:42.978033 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245605ea-af8f-487a-9362-ec3605ab2cce admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:42.998528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168ce72-bced-4c16-86b1-13e696a53a5c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:43.003399 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1139/2243] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:43.013784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168ce72-bced-4c16-86b1-13e696a53a5c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:43.020843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e3ffd-de88-4bca-ac03-38ea52bf8dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:43.021540 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e3ffd-de88-4bca-ac03-38ea52bf8dbe admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:43.022450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e3ffd-de88-4bca-ac03-38ea52bf8dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:43.022738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e3ffd-de88-4bca-ac03-38ea52bf8dbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:43.023441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e3ffd-de88-4bca-ac03-38ea52bf8dbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:43.025560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e3ffd-de88-4bca-ac03-38ea52bf8dbe admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:43.070260 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1105/2244] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:42 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:43.074351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245605ea-af8f-487a-9362-ec3605ab2cce admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:43.099900 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245605ea-af8f-487a-9362-ec3605ab2cce admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:43.210407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316b777-d701-4abd-8be5-3101f7b70eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:43.211157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316b777-d701-4abd-8be5-3101f7b70eed admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:43.211442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316b777-d701-4abd-8be5-3101f7b70eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:43.211713 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316b777-d701-4abd-8be5-3101f7b70eed admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:43.212509 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316b777-d701-4abd-8be5-3101f7b70eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:43.215788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316b777-d701-4abd-8be5-3101f7b70eed admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:43.247817 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1140/2245] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:43 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:43.253180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e3ffd-de88-4bca-ac03-38ea52bf8dbe admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:43.279155 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e3ffd-de88-4bca-ac03-38ea52bf8dbe admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:43.285911 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824d2a3-409d-410e-b811-812eb0e36089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:43.288317 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824d2a3-409d-410e-b811-812eb0e36089 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:43.288680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824d2a3-409d-410e-b811-812eb0e36089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:43.289020 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824d2a3-409d-410e-b811-812eb0e36089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:43.291140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824d2a3-409d-410e-b811-812eb0e36089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:43.294251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824d2a3-409d-410e-b811-812eb0e36089 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:43.315832 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1106/2246] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:43.319966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316b777-d701-4abd-8be5-3101f7b70eed admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:43.331574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316b777-d701-4abd-8be5-3101f7b70eed admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:43.336897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222d257-2c30-4936-9ebf-eb41d09c166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:43.338266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222d257-2c30-4936-9ebf-eb41d09c166f admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:43.338266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222d257-2c30-4936-9ebf-eb41d09c166f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:43.338513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222d257-2c30-4936-9ebf-eb41d09c166f admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:43.339595 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222d257-2c30-4936-9ebf-eb41d09c166f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:43.341723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222d257-2c30-4936-9ebf-eb41d09c166f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:43.357069 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1141/2247] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:43.359494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824d2a3-409d-410e-b811-812eb0e36089 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:43.360413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222d257-2c30-4936-9ebf-eb41d09c166f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:43.362536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b222d257-2c30-4936-9ebf-eb41d09c166f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:43.370647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824d2a3-409d-410e-b811-812eb0e36089 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:43.387770 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222d257-2c30-4936-9ebf-eb41d09c166f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:43.390653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0cd87f-6ce2-4fda-aad5-cd5e143e1f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:43.392003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cd87f-6ce2-4fda-aad5-cd5e143e1f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:43.392501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cd87f-6ce2-4fda-aad5-cd5e143e1f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:43.392930 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cd87f-6ce2-4fda-aad5-cd5e143e1f81 admin admin] PATH_INFO: `/v3/projects/23b7751c7de64a55be87643acb50ae52/users/fa91f5c13dda429ebfdd77325d9cf928/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:43.394578 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:43.395599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:43.404457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:43.404759 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] PATH_INFO: `/v3/projects/0d63971c6cea4b5db874259b474d983e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:43.412842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d63971c6cea4b5db874259b474d983e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:43.414849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:43.446796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0cd87f-6ce2-4fda-aad5-cd5e143e1f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b7751c7de64a55be87643acb50ae52, user_id=fa91f5c13dda429ebfdd77325d9cf928, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:43.449960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0cd87f-6ce2-4fda-aad5-cd5e143e1f81 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:43.489708 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1107/2248] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:43 2022] PUT /identity/v3/projects/23b7751c7de64a55be87643acb50ae52/users/fa91f5c13dda429ebfdd77325d9cf928/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:43.492793 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de8437-cb13-4fba-808f-f550fe5d92a3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:43.493096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de8437-cb13-4fba-808f-f550fe5d92a3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:43.493378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de8437-cb13-4fba-808f-f550fe5d92a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:43.504747 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:43.518506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:43.533463 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d63971c6cea4b5db874259b474d983e', 'request_id': 'req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:43.550413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d63971c6cea4b5db874259b474d983e', 'request_id': 'req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:43.564916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-81de8437-cb13-4fba-808f-f550fe5d92a3 None None] MFA Rules not processed for user `fa91f5c13dda429ebfdd77325d9cf928`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:43.566926 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-81de8437-cb13-4fba-808f-f550fe5d92a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:43.569930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d63971c6cea4b5db874259b474d983e', 'request_id': 'req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:43.592687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] The token cache is being invalidate because project 0d63971c6cea4b5db874259b474d983e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:43.592986 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:43.594020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:43.596969 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1142/2249] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:43 2022] DELETE /identity/v3/projects/0d63971c6cea4b5db874259b474d983e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:43.601374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:43.628826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:43.726877 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1108/2250] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:43.739929 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de8437-cb13-4fba-808f-f550fe5d92a3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:43.741307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-81de8437-cb13-4fba-808f-f550fe5d92a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:43.780844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3d34f-2c3d-4d42-bce4-02d46befcbad admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:43.795142 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0290193-ef4f-48e3-98dd-5c56fdb79620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:43.796531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0290193-ef4f-48e3-98dd-5c56fdb79620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:43.796809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0290193-ef4f-48e3-98dd-5c56fdb79620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:43.797090 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0290193-ef4f-48e3-98dd-5c56fdb79620 admin admin] PATH_INFO: `/v3/users/d8bc6a064ed3421e8eefd11423a6fe82` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:43.801283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0290193-ef4f-48e3-98dd-5c56fdb79620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8bc6a064ed3421e8eefd11423a6fe82)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:43.803810 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0290193-ef4f-48e3-98dd-5c56fdb79620 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:43.815307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de8437-cb13-4fba-808f-f550fe5d92a3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:43.819947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118045d-c195-4d8e-b0e3-3c890c23301c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:43.820787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118045d-c195-4d8e-b0e3-3c890c23301c service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:43.820787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118045d-c195-4d8e-b0e3-3c890c23301c service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:43.822328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118045d-c195-4d8e-b0e3-3c890c23301c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:43.839406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118045d-c195-4d8e-b0e3-3c890c23301c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:43.841226 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118045d-c195-4d8e-b0e3-3c890c23301c service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:43.858044 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1109/2251] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:43.911288 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d0290193-ef4f-48e3-98dd-5c56fdb79620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8bc6a064ed3421e8eefd11423a6fe82'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:43.912080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d0290193-ef4f-48e3-98dd-5c56fdb79620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8bc6a064ed3421e8eefd11423a6fe82'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:43.925968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d0290193-ef4f-48e3-98dd-5c56fdb79620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8bc6a064ed3421e8eefd11423a6fe82'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:43.932943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d0290193-ef4f-48e3-98dd-5c56fdb79620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8bc6a064ed3421e8eefd11423a6fe82'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:43.933846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d0290193-ef4f-48e3-98dd-5c56fdb79620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8bc6a064ed3421e8eefd11423a6fe82'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:43.954163 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1143/2252] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:43 2022] DELETE /identity/v3/users/d8bc6a064ed3421e8eefd11423a6fe82 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:44.412974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118045d-c195-4d8e-b0e3-3c890c23301c service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:44.443460 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118045d-c195-4d8e-b0e3-3c890c23301c service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:44.636561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:44.637569 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:44.638787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:44.640116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] PATH_INFO: `/v3/projects/72727482fc914018b76fb68789eb4b03` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:44.646213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72727482fc914018b76fb68789eb4b03)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:44.652776 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:44.730289 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:44.748115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72727482fc914018b76fb68789eb4b03', 'request_id': 'req-6715174f-bde2-40f9-8010-552382328356'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:44.766919 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72727482fc914018b76fb68789eb4b03', 'request_id': 'req-6715174f-bde2-40f9-8010-552382328356'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:44.778500 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72727482fc914018b76fb68789eb4b03', 'request_id': 'req-6715174f-bde2-40f9-8010-552382328356'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:44.811757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] The token cache is being invalidate because project 72727482fc914018b76fb68789eb4b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:44.813224 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:44.815643 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6715174f-bde2-40f9-8010-552382328356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:44.820129 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1110/2253] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:44 2022] DELETE /identity/v3/projects/72727482fc914018b76fb68789eb4b03 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:44.825328 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc0dfd-7ad1-4a0b-a1fc-425a0a6ac538 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:44.827570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc0dfd-7ad1-4a0b-a1fc-425a0a6ac538 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:44.828001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc0dfd-7ad1-4a0b-a1fc-425a0a6ac538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:44.859493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-72dc0dfd-7ad1-4a0b-a1fc-425a0a6ac538 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:44.861180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-72dc0dfd-7ad1-4a0b-a1fc-425a0a6ac538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:44.912921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:44.944738 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1144/2254] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:44.952782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d4331-a2c8-4545-bedd-f2e87a31023c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:44.953068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d4331-a2c8-4545-bedd-f2e87a31023c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:44.953343 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d4331-a2c8-4545-bedd-f2e87a31023c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:44.988207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-796d4331-a2c8-4545-bedd-f2e87a31023c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:44.992585 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-796d4331-a2c8-4545-bedd-f2e87a31023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:45.012288 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1111/2255] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:45.013420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc0dfd-7ad1-4a0b-a1fc-425a0a6ac538 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:45.024767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc0dfd-7ad1-4a0b-a1fc-425a0a6ac538 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:45.034907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f423f-37cb-429b-91cf-a55474872f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:45.036051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f423f-37cb-429b-91cf-a55474872f5d admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:45.036051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f423f-37cb-429b-91cf-a55474872f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:45.036244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f423f-37cb-429b-91cf-a55474872f5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:45.036956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f423f-37cb-429b-91cf-a55474872f5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:45.037277 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f423f-37cb-429b-91cf-a55474872f5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:45.046708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f423f-37cb-429b-91cf-a55474872f5d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:45.061413 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1145/2256] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:45.066103 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d4331-a2c8-4545-bedd-f2e87a31023c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:45.080864 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d4331-a2c8-4545-bedd-f2e87a31023c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:45.092855 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb51f17-7ca7-4e99-8509-2dd33b14113f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:45.093835 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51f17-7ca7-4e99-8509-2dd33b14113f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:45.094287 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51f17-7ca7-4e99-8509-2dd33b14113f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:45.094748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51f17-7ca7-4e99-8509-2dd33b14113f admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:45.097146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb51f17-7ca7-4e99-8509-2dd33b14113f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:45.099436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb51f17-7ca7-4e99-8509-2dd33b14113f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:45.172324 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1112/2257] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:45 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:45.179361 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f423f-37cb-429b-91cf-a55474872f5d admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:45.194494 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f423f-37cb-429b-91cf-a55474872f5d admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:45.352754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4286b82-785b-440f-b07a-04773ceeb954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:45.353522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4286b82-785b-440f-b07a-04773ceeb954 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:45.353795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4286b82-785b-440f-b07a-04773ceeb954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:45.354070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4286b82-785b-440f-b07a-04773ceeb954 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:45.354996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4286b82-785b-440f-b07a-04773ceeb954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:45.357623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4286b82-785b-440f-b07a-04773ceeb954 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:45.388407 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1146/2258] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:45 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:45.392130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb51f17-7ca7-4e99-8509-2dd33b14113f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:45.413027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb51f17-7ca7-4e99-8509-2dd33b14113f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:45.436095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b9794-0829-431e-b059-a38593e4899d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:45.436952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9794-0829-431e-b059-a38593e4899d admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:45.437319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9794-0829-431e-b059-a38593e4899d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:45.437680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9794-0829-431e-b059-a38593e4899d admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:45.438575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b9794-0829-431e-b059-a38593e4899d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:45.440405 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b9794-0829-431e-b059-a38593e4899d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:45.469676 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1113/2259] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:45.470774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4286b82-785b-440f-b07a-04773ceeb954 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:45.487221 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4286b82-785b-440f-b07a-04773ceeb954 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:45.495831 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf80f4-ca93-4a7e-b52b-76f1e1ca6918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:45.496847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf80f4-ca93-4a7e-b52b-76f1e1ca6918 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:45.497715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf80f4-ca93-4a7e-b52b-76f1e1ca6918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:45.497715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf80f4-ca93-4a7e-b52b-76f1e1ca6918 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:45.498132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf80f4-ca93-4a7e-b52b-76f1e1ca6918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:45.499900 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf80f4-ca93-4a7e-b52b-76f1e1ca6918 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:45.531249 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1147/2260] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:45.535586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b9794-0829-431e-b059-a38593e4899d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:45.549358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b9794-0829-431e-b059-a38593e4899d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:45.555156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565afa52-c947-4d46-ae3b-b1714c2240f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:45.556295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565afa52-c947-4d46-ae3b-b1714c2240f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:45.556586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565afa52-c947-4d46-ae3b-b1714c2240f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:45.556872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565afa52-c947-4d46-ae3b-b1714c2240f1 admin admin] PATH_INFO: `/v3/projects/26010585269545dbb21f9e0a46b0a7ce/users/38b4670b68a24b3586f139282cbb609c/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:45.575799 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565afa52-c947-4d46-ae3b-b1714c2240f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26010585269545dbb21f9e0a46b0a7ce, user_id=38b4670b68a24b3586f139282cbb609c, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:45.578761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565afa52-c947-4d46-ae3b-b1714c2240f1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:45.603542 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1114/2261] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:45 2022] PUT /identity/v3/projects/26010585269545dbb21f9e0a46b0a7ce/users/38b4670b68a24b3586f139282cbb609c/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:45.617850 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4833d-555f-416c-90cc-f230a73291fe None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:45.618144 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4833d-555f-416c-90cc-f230a73291fe None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:45.618542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4833d-555f-416c-90cc-f230a73291fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:45.659491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-baa4833d-555f-416c-90cc-f230a73291fe None None] MFA Rules not processed for user `38b4670b68a24b3586f139282cbb609c`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:45.661006 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-baa4833d-555f-416c-90cc-f230a73291fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:45.765242 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1148/2262] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:45.765800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565afa52-c947-4d46-ae3b-b1714c2240f1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:45.849110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565afa52-c947-4d46-ae3b-b1714c2240f1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:45.860124 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2a79-c585-41b1-98a9-144d5ecf11b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:45.860836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2a79-c585-41b1-98a9-144d5ecf11b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:45.861133 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2a79-c585-41b1-98a9-144d5ecf11b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:45.861424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2a79-c585-41b1-98a9-144d5ecf11b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:45.862162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d2a79-c585-41b1-98a9-144d5ecf11b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:45.865269 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d2a79-c585-41b1-98a9-144d5ecf11b2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:45.914090 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1115/2263] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:45 2022] POST /identity/v3/projects => generated 367 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:45.915370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4833d-555f-416c-90cc-f230a73291fe None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:45.930787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4833d-555f-416c-90cc-f230a73291fe None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:46.038093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:46.114481 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5c0f2-f430-429d-a51b-bf918f73072a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:46.115204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5c0f2-f430-429d-a51b-bf918f73072a admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:46.117167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5c0f2-f430-429d-a51b-bf918f73072a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:46.117470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5c0f2-f430-429d-a51b-bf918f73072a admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:46.118379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5c0f2-f430-429d-a51b-bf918f73072a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:46.120359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5c0f2-f430-429d-a51b-bf918f73072a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:46.191441 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1149/2264] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:45 2022] POST /identity/v3/users => generated 290 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:46.193651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2a79-c585-41b1-98a9-144d5ecf11b2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:46.219339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2a79-c585-41b1-98a9-144d5ecf11b2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:46.233439 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b3340-c5fe-4187-8cf1-7ce62b9aaf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:46.234618 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b3340-c5fe-4187-8cf1-7ce62b9aaf23 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:46.235050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b3340-c5fe-4187-8cf1-7ce62b9aaf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:46.235474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b3340-c5fe-4187-8cf1-7ce62b9aaf23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:46.236404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b3340-c5fe-4187-8cf1-7ce62b9aaf23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:46.239642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b3340-c5fe-4187-8cf1-7ce62b9aaf23 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:46.271021 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1116/2265] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:46.275436 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5c0f2-f430-429d-a51b-bf918f73072a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:46.312395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5c0f2-f430-429d-a51b-bf918f73072a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:46.320470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ae0b2-a296-4580-b355-8ed9165e5fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:46.321531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ae0b2-a296-4580-b355-8ed9165e5fab admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:46.321817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ae0b2-a296-4580-b355-8ed9165e5fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:46.322111 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ae0b2-a296-4580-b355-8ed9165e5fab admin admin] PATH_INFO: `/v3/projects/8bdee6aaf1b74a53b265fc05bd2c50eb/users/11b17893de284ab2821f033581e2fa64/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:46.341562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ae0b2-a296-4580-b355-8ed9165e5fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdee6aaf1b74a53b265fc05bd2c50eb, user_id=11b17893de284ab2821f033581e2fa64, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:46.344265 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ae0b2-a296-4580-b355-8ed9165e5fab admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:46.363661 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1150/2266] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:46 2022] PUT /identity/v3/projects/8bdee6aaf1b74a53b265fc05bd2c50eb/users/11b17893de284ab2821f033581e2fa64/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:46.365186 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b3340-c5fe-4187-8cf1-7ce62b9aaf23 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:46.378481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b3340-c5fe-4187-8cf1-7ce62b9aaf23 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:46.442781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:46.480878 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0672e6-a90e-4e72-a8ab-8c5bd3467995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:46.481817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0672e6-a90e-4e72-a8ab-8c5bd3467995 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:46.482207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0672e6-a90e-4e72-a8ab-8c5bd3467995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:46.482695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0672e6-a90e-4e72-a8ab-8c5bd3467995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:46.483604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0672e6-a90e-4e72-a8ab-8c5bd3467995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:46.485881 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0672e6-a90e-4e72-a8ab-8c5bd3467995 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:46.502260 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1117/2267] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:46.506581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ae0b2-a296-4580-b355-8ed9165e5fab admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:46.526429 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ae0b2-a296-4580-b355-8ed9165e5fab admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:46.533965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb8626-2112-4ab3-aa28-839d91907563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:46.535365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb8626-2112-4ab3-aa28-839d91907563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:46.535932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb8626-2112-4ab3-aa28-839d91907563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:46.536417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb8626-2112-4ab3-aa28-839d91907563 admin admin] PATH_INFO: `/v3/projects/8bdee6aaf1b74a53b265fc05bd2c50eb/users/11b17893de284ab2821f033581e2fa64/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:46.538640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb8626-2112-4ab3-aa28-839d91907563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdee6aaf1b74a53b265fc05bd2c50eb, user_id=11b17893de284ab2821f033581e2fa64, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:46.541542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb8626-2112-4ab3-aa28-839d91907563 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:46.559278 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1151/2268] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:46 2022] PUT /identity/v3/projects/8bdee6aaf1b74a53b265fc05bd2c50eb/users/11b17893de284ab2821f033581e2fa64/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:46.571497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c1a81-e1f4-44bc-b414-002322c3507c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:46.572071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c1a81-e1f4-44bc-b414-002322c3507c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:46.572515 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c1a81-e1f4-44bc-b414-002322c3507c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:46.614859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7d3c1a81-e1f4-44bc-b414-002322c3507c None None] MFA Rules not processed for user `11b17893de284ab2821f033581e2fa64`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:46.616843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7d3c1a81-e1f4-44bc-b414-002322c3507c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:46.771134 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1118/2269] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:46.842647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb8626-2112-4ab3-aa28-839d91907563 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:46.844723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-35cb8626-2112-4ab3-aa28-839d91907563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:46.927516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb8626-2112-4ab3-aa28-839d91907563 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:46.931714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee750919-60f3-494d-b574-73c0bf16081c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:46.932489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee750919-60f3-494d-b574-73c0bf16081c service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:46.932826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee750919-60f3-494d-b574-73c0bf16081c service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:46.933194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee750919-60f3-494d-b574-73c0bf16081c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:46.948239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee750919-60f3-494d-b574-73c0bf16081c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:46.950916 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee750919-60f3-494d-b574-73c0bf16081c service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:46.974150 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1152/2270] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:47.064214 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c1a81-e1f4-44bc-b414-002322c3507c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:47.077385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c1a81-e1f4-44bc-b414-002322c3507c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:47.082071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873576fe-b08f-4442-b026-8ff8273563ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:47.083763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873576fe-b08f-4442-b026-8ff8273563ae service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:47.084347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873576fe-b08f-4442-b026-8ff8273563ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:47.084869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873576fe-b08f-4442-b026-8ff8273563ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:47.165533 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873576fe-b08f-4442-b026-8ff8273563ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:47.167675 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873576fe-b08f-4442-b026-8ff8273563ae service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:47.180267 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1119/2271] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:47.482303 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee750919-60f3-494d-b574-73c0bf16081c service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:47.485469 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ee750919-60f3-494d-b574-73c0bf16081c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:47.605293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee750919-60f3-494d-b574-73c0bf16081c service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:47.618323 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:47.619767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:47.621346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:47.622039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] PATH_INFO: `/v3/users/fa91f5c13dda429ebfdd77325d9cf928` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:47.624506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa91f5c13dda429ebfdd77325d9cf928)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:47.628512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:47.701523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:47.735671 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa91f5c13dda429ebfdd77325d9cf928'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:47.738907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa91f5c13dda429ebfdd77325d9cf928'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:47.845636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa91f5c13dda429ebfdd77325d9cf928'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:47.870774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa91f5c13dda429ebfdd77325d9cf928'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:47.872642 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa91f5c13dda429ebfdd77325d9cf928'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:47.916704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491def-ee94-4bc1-97ed-7f25c84b92a0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:47.916995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491def-ee94-4bc1-97ed-7f25c84b92a0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:47.917438 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491def-ee94-4bc1-97ed-7f25c84b92a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:47.919472 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1153/2272] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:47 2022] DELETE /identity/v3/users/fa91f5c13dda429ebfdd77325d9cf928 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:47.948391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c0491def-ee94-4bc1-97ed-7f25c84b92a0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:47.950147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c0491def-ee94-4bc1-97ed-7f25c84b92a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:48.038258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:48.048038 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1120/2273] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:48.096531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:48.101714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:48.228856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52efd4-3ead-4260-8267-9622f6bfa866 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:48.244618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dccc6c-fd53-471b-b193-9823761948a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:48.254832 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dccc6c-fd53-471b-b193-9823761948a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:48.255570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dccc6c-fd53-471b-b193-9823761948a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:48.255787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dccc6c-fd53-471b-b193-9823761948a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:48.274414 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dccc6c-fd53-471b-b193-9823761948a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:48.276395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dccc6c-fd53-471b-b193-9823761948a4 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:48.314699 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1154/2274] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:49.011240 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0491def-ee94-4bc1-97ed-7f25c84b92a0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:49.012738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c0491def-ee94-4bc1-97ed-7f25c84b92a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:49.042868 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0491def-ee94-4bc1-97ed-7f25c84b92a0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:49.072019 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:49.078928 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.079580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:49.080057 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] PATH_INFO: `/v3/projects/23b7751c7de64a55be87643acb50ae52` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:49.086152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] RBAC: Authorizing `identity:delete_project(project_id=23b7751c7de64a55be87643acb50ae52)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:49.088684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:49.268703 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23b7751c7de64a55be87643acb50ae52', 'request_id': 'req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:49.285684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23b7751c7de64a55be87643acb50ae52', 'request_id': 'req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:49.308996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23b7751c7de64a55be87643acb50ae52', 'request_id': 'req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:49.335494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] The token cache is being invalidate because project 23b7751c7de64a55be87643acb50ae52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:49.340132 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:49.341324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5fc23adb-73ad-45b4-b0e3-ab706ef4d2af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:49.344388 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1121/2275] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:49 2022] DELETE /identity/v3/projects/23b7751c7de64a55be87643acb50ae52 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:49.349127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4ce5a-64d9-41d4-a77f-83da21d32699 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.349986 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4ce5a-64d9-41d4-a77f-83da21d32699 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:49.350453 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4ce5a-64d9-41d4-a77f-83da21d32699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:49.390828 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-cbd4ce5a-64d9-41d4-a77f-83da21d32699 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:49.392212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-cbd4ce5a-64d9-41d4-a77f-83da21d32699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:49.452318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1400fa-c972-4b6c-aa57-b44555e802d3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.454543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1400fa-c972-4b6c-aa57-b44555e802d3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:49.455028 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1400fa-c972-4b6c-aa57-b44555e802d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:49.482774 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ef1400fa-c972-4b6c-aa57-b44555e802d3 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:49.484349 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ef1400fa-c972-4b6c-aa57-b44555e802d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:49.507349 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1155/2276] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:49.515309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae176368-c749-43a1-8a10-db3aadfce5e8 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.516722 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae176368-c749-43a1-8a10-db3aadfce5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:49.517096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae176368-c749-43a1-8a10-db3aadfce5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:49.545096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ae176368-c749-43a1-8a10-db3aadfce5e8 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:49.546571 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ae176368-c749-43a1-8a10-db3aadfce5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:49.557955 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1156/2277] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:49.562180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae176368-c749-43a1-8a10-db3aadfce5e8 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:49.563909 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1122/2278] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:13:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:49.572204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae176368-c749-43a1-8a10-db3aadfce5e8 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:49.578156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580b902-4f97-453f-b2e3-bd3fb1ed1614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:49.578677 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580b902-4f97-453f-b2e3-bd3fb1ed1614 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.579025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580b902-4f97-453f-b2e3-bd3fb1ed1614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:49.579363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580b902-4f97-453f-b2e3-bd3fb1ed1614 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:49.580217 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580b902-4f97-453f-b2e3-bd3fb1ed1614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:49.580896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580b902-4f97-453f-b2e3-bd3fb1ed1614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:49.582549 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580b902-4f97-453f-b2e3-bd3fb1ed1614 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:49.600963 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1157/2279] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:49.602580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1400fa-c972-4b6c-aa57-b44555e802d3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:49.614785 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580b902-4f97-453f-b2e3-bd3fb1ed1614 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:49.617251 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7580b902-4f97-453f-b2e3-bd3fb1ed1614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:49.621126 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1400fa-c972-4b6c-aa57-b44555e802d3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:49.628001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe36d-66bd-40f6-9435-7b8e75e83d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:49.631510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe36d-66bd-40f6-9435-7b8e75e83d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.631510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe36d-66bd-40f6-9435-7b8e75e83d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:49.631510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe36d-66bd-40f6-9435-7b8e75e83d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:49.631510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fe36d-66bd-40f6-9435-7b8e75e83d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:49.632642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fe36d-66bd-40f6-9435-7b8e75e83d36 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:49.666286 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1123/2280] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:49 2022] POST /identity/v3/projects => generated 389 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:49.670669 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe36d-66bd-40f6-9435-7b8e75e83d36 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:49.698552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:49.735116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580b902-4f97-453f-b2e3-bd3fb1ed1614 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:49.762162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe36d-66bd-40f6-9435-7b8e75e83d36 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:49.771705 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a35ad-04a1-4553-81b6-291e679d216c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:49.773010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a35ad-04a1-4553-81b6-291e679d216c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.773309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a35ad-04a1-4553-81b6-291e679d216c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:49.773744 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a35ad-04a1-4553-81b6-291e679d216c admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:49.774946 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a35ad-04a1-4553-81b6-291e679d216c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:49.779414 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a35ad-04a1-4553-81b6-291e679d216c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:49.795949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:49.809707 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1124/2281] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:49 2022] POST /identity/v3/users => generated 295 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:49.811441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26734ad-1e7d-4b8d-8864-198acce62cd6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.811738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26734ad-1e7d-4b8d-8864-198acce62cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:49.812043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26734ad-1e7d-4b8d-8864-198acce62cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:49.824025 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c167ef5-3c37-48b6-9581-848781b36c0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:49.824712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c167ef5-3c37-48b6-9581-848781b36c0b service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.825000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c167ef5-3c37-48b6-9581-848781b36c0b service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:49.825284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c167ef5-3c37-48b6-9581-848781b36c0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:49.845446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a26734ad-1e7d-4b8d-8864-198acce62cd6 None None] MFA Rules not processed for user `ae9cd6d6d5824aa1932d3353578285af`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:49.846765 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a26734ad-1e7d-4b8d-8864-198acce62cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:49.889173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c167ef5-3c37-48b6-9581-848781b36c0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:49.890811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c167ef5-3c37-48b6-9581-848781b36c0b service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:49.900251 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1158/2282] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:13:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:13:49.901071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c167ef5-3c37-48b6-9581-848781b36c0b service swift] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:49.911066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c167ef5-3c37-48b6-9581-848781b36c0b service swift] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:49.916957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9c7d1-2217-4b09-a01e-fe723ca2ad88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:49.917617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9c7d1-2217-4b09-a01e-fe723ca2ad88 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.917901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9c7d1-2217-4b09-a01e-fe723ca2ad88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:49.918014 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1125/2283] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:49.918173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9c7d1-2217-4b09-a01e-fe723ca2ad88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:49.918937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9c7d1-2217-4b09-a01e-fe723ca2ad88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:49.920712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9c7d1-2217-4b09-a01e-fe723ca2ad88 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:49.932383 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1159/2284] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:49.932882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26734ad-1e7d-4b8d-8864-198acce62cd6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:49.934294 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a26734ad-1e7d-4b8d-8864-198acce62cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:49.936142 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9c7d1-2217-4b09-a01e-fe723ca2ad88 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:49.947053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9c7d1-2217-4b09-a01e-fe723ca2ad88 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:49.952789 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f45ec-a212-4825-9110-d260ad60b1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:49.953645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f45ec-a212-4825-9110-d260ad60b1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.954030 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f45ec-a212-4825-9110-d260ad60b1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:49.954513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f45ec-a212-4825-9110-d260ad60b1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:49.955534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f45ec-a212-4825-9110-d260ad60b1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:49.957499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f45ec-a212-4825-9110-d260ad60b1e6 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:49.969415 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1160/2285] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:49.973946 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f45ec-a212-4825-9110-d260ad60b1e6 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:49.987160 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f45ec-a212-4825-9110-d260ad60b1e6 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:49.996885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6fa50-1b59-4919-a0e4-d538e9768c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:49.998271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6fa50-1b59-4919-a0e4-d538e9768c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:49.998566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6fa50-1b59-4919-a0e4-d538e9768c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:50.000219 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6fa50-1b59-4919-a0e4-d538e9768c9e admin admin] PATH_INFO: `/v3/projects/07e46e591bba42b1acaf36faface1f0c/users/8603d5ce5ab840f7948fd111108e65ac/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:50.032351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6fa50-1b59-4919-a0e4-d538e9768c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=07e46e591bba42b1acaf36faface1f0c, user_id=8603d5ce5ab840f7948fd111108e65ac, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:50.035588 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6fa50-1b59-4919-a0e4-d538e9768c9e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:50.053084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26734ad-1e7d-4b8d-8864-198acce62cd6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:50.060917 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1161/2286] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:49 2022] PUT /identity/v3/projects/07e46e591bba42b1acaf36faface1f0c/users/8603d5ce5ab840f7948fd111108e65ac/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:50.075624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74734634-d650-4e09-947c-5afd647f5226 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:50.076113 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74734634-d650-4e09-947c-5afd647f5226 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:50.076639 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74734634-d650-4e09-947c-5afd647f5226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:50.114845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-74734634-d650-4e09-947c-5afd647f5226 None None] MFA Rules not processed for user `8603d5ce5ab840f7948fd111108e65ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:50.118068 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-74734634-d650-4e09-947c-5afd647f5226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:50.150869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d63803-a6f2-4b92-98a3-a48de439c95b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:50.151564 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:50.152055 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d63803-a6f2-4b92-98a3-a48de439c95b service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:50.152529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d63803-a6f2-4b92-98a3-a48de439c95b service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:50.153318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d63803-a6f2-4b92-98a3-a48de439c95b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:50.176965 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:50.242659 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d63803-a6f2-4b92-98a3-a48de439c95b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:50.244648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d63803-a6f2-4b92-98a3-a48de439c95b service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:50.252781 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1162/2287] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:50.264516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74734634-d650-4e09-947c-5afd647f5226 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:50.266069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-74734634-d650-4e09-947c-5afd647f5226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:50.271048 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1126/2288] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:13:50.350646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74734634-d650-4e09-947c-5afd647f5226 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:50.355780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c3a11-572c-41bc-a9dd-8cb040946e93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:50.356455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c3a11-572c-41bc-a9dd-8cb040946e93 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:50.356801 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c3a11-572c-41bc-a9dd-8cb040946e93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:50.357164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c3a11-572c-41bc-a9dd-8cb040946e93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:50.372121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c3a11-572c-41bc-a9dd-8cb040946e93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:50.373844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c3a11-572c-41bc-a9dd-8cb040946e93 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:50.389715 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1163/2289] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:50.732236 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb42701-69a9-4466-bea4-fd763f33702d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:50.732679 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb42701-69a9-4466-bea4-fd763f33702d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:50.733085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb42701-69a9-4466-bea4-fd763f33702d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:50.786497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-abb42701-69a9-4466-bea4-fd763f33702d None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:50.805928 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-abb42701-69a9-4466-bea4-fd763f33702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:50.947836 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1127/2290] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jul 5 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:50.966190 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c3a11-572c-41bc-a9dd-8cb040946e93 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:51.146211 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c3a11-572c-41bc-a9dd-8cb040946e93 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:51.153166 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96dd758-5af4-489b-bfff-39b390b1a3fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:51.153879 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96dd758-5af4-489b-bfff-39b390b1a3fb service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:51.154210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96dd758-5af4-489b-bfff-39b390b1a3fb service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:51.154527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96dd758-5af4-489b-bfff-39b390b1a3fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:51.165643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96dd758-5af4-489b-bfff-39b390b1a3fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:51.167570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96dd758-5af4-489b-bfff-39b390b1a3fb service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:51.185516 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1164/2291] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:13:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:13:51.649050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e41537-2071-4198-83db-81bc0b653c26 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:51.649657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e41537-2071-4198-83db-81bc0b653c26 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:51.649657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e41537-2071-4198-83db-81bc0b653c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:51.708110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c4e41537-2071-4198-83db-81bc0b653c26 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:51.712948 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c4e41537-2071-4198-83db-81bc0b653c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:51.731512 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1128/2292] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:13:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:51.753300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96dd758-5af4-489b-bfff-39b390b1a3fb service swift] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:51.765191 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96dd758-5af4-489b-bfff-39b390b1a3fb service swift] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:51.771655 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424488b8-0d85-43ab-9013-7feb6f082add service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:51.773714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424488b8-0d85-43ab-9013-7feb6f082add service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:51.774165 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424488b8-0d85-43ab-9013-7feb6f082add service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:51.774614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424488b8-0d85-43ab-9013-7feb6f082add service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:51.784692 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424488b8-0d85-43ab-9013-7feb6f082add service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:51.786651 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424488b8-0d85-43ab-9013-7feb6f082add service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:51.811759 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1165/2293] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:13:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:13:52.347092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898d09f-b42b-4a29-8fb3-3ba4f73e63d5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:52.347800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898d09f-b42b-4a29-8fb3-3ba4f73e63d5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:52.348268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898d09f-b42b-4a29-8fb3-3ba4f73e63d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:52.413554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4898d09f-b42b-4a29-8fb3-3ba4f73e63d5 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:52.420264 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4898d09f-b42b-4a29-8fb3-3ba4f73e63d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:52.452432 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1129/2294] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jul 5 10:13:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:52.475922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424488b8-0d85-43ab-9013-7feb6f082add service swift] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:52.499360 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424488b8-0d85-43ab-9013-7feb6f082add service swift] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:52.518282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1488a6-636c-4388-b170-8dcdfda26ef8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:52.519262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1488a6-636c-4388-b170-8dcdfda26ef8 service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:52.520291 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1488a6-636c-4388-b170-8dcdfda26ef8 service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:52.520644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1488a6-636c-4388-b170-8dcdfda26ef8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:52.541371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1488a6-636c-4388-b170-8dcdfda26ef8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:52.542888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1488a6-636c-4388-b170-8dcdfda26ef8 service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:52.561876 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1166/2295] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:13:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:13:52.632916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb16b02-92f3-4769-a173-4cc561b13bbc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:52.633212 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb16b02-92f3-4769-a173-4cc561b13bbc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:52.633586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb16b02-92f3-4769-a173-4cc561b13bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:52.674978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ecb16b02-92f3-4769-a173-4cc561b13bbc None None] MFA Rules not processed for user `ae9cd6d6d5824aa1932d3353578285af`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:52.695423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ecb16b02-92f3-4769-a173-4cc561b13bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:52.746040 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1130/2296] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:54.225128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1488a6-636c-4388-b170-8dcdfda26ef8 service swift] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:54.226965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-aa1488a6-636c-4388-b170-8dcdfda26ef8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:54.319555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:54.381446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1488a6-636c-4388-b170-8dcdfda26ef8 service swift] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:54.388642 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:54.389815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:54.390322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:54.390674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] PATH_INFO: `/v3/users/38b4670b68a24b3586f139282cbb609c` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:54.392203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38b4670b68a24b3586f139282cbb609c)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:54.394748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:54.484731 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38b4670b68a24b3586f139282cbb609c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:54.489015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38b4670b68a24b3586f139282cbb609c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:54.532508 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38b4670b68a24b3586f139282cbb609c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:54.541560 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38b4670b68a24b3586f139282cbb609c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:54.543610 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38b4670b68a24b3586f139282cbb609c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:54.575125 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1167/2297] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:54 2022] DELETE /identity/v3/users/38b4670b68a24b3586f139282cbb609c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:54.577141 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e2987-897f-45ed-a632-e6511d600ee4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:54.577596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e2987-897f-45ed-a632-e6511d600ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:54.577791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e2987-897f-45ed-a632-e6511d600ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:54.606122 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-030e2987-897f-45ed-a632-e6511d600ee4 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:54.607821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-030e2987-897f-45ed-a632-e6511d600ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:54.782262 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1131/2298] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:54.783632 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:54.785122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:54.866614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b26b54-6a23-48ed-b1cd-312efdab6ad7 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:54.871161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8f02e-084c-40e1-b443-b49d002cf3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:54.872013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8f02e-084c-40e1-b443-b49d002cf3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:54.872383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8f02e-084c-40e1-b443-b49d002cf3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:54.872745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8f02e-084c-40e1-b443-b49d002cf3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:54.884592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8f02e-084c-40e1-b443-b49d002cf3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:54.886476 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8f02e-084c-40e1-b443-b49d002cf3a1 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:54.909650 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1168/2299] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:55.336178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e2987-897f-45ed-a632-e6511d600ee4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:55.338889 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-030e2987-897f-45ed-a632-e6511d600ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:55.359172 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e2987-897f-45ed-a632-e6511d600ee4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:55.364461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:55.365361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:55.365634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:55.365909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] PATH_INFO: `/v3/projects/26010585269545dbb21f9e0a46b0a7ce` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:55.367304 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26010585269545dbb21f9e0a46b0a7ce)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:55.375319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:55.398741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8f02e-084c-40e1-b443-b49d002cf3a1 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:55.403736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-31b8f02e-084c-40e1-b443-b49d002cf3a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:55.423381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8f02e-084c-40e1-b443-b49d002cf3a1 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:55.434125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:55.441506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26010585269545dbb21f9e0a46b0a7ce', 'request_id': 'req-bedabfcf-1cea-465e-9116-d253cf35dc61'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.451633 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26010585269545dbb21f9e0a46b0a7ce', 'request_id': 'req-bedabfcf-1cea-465e-9116-d253cf35dc61'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.459341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26010585269545dbb21f9e0a46b0a7ce', 'request_id': 'req-bedabfcf-1cea-465e-9116-d253cf35dc61'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.472911 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] The token cache is being invalidate because project 26010585269545dbb21f9e0a46b0a7ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:55.473295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.474115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.475781 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1132/2300] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:55 2022] DELETE /identity/v3/projects/26010585269545dbb21f9e0a46b0a7ce => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:55.479781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:55.481205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:55.481972 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:55.493443 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929c12a-fa4a-48d6-b81e-f8bccd4f99c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:55.494429 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929c12a-fa4a-48d6-b81e-f8bccd4f99c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:55.494919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929c12a-fa4a-48d6-b81e-f8bccd4f99c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:55.495526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929c12a-fa4a-48d6-b81e-f8bccd4f99c5 admin admin] PATH_INFO: `/v3/users/bb41d8ee37f8426d9ac14efbbda441a5` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:55.496997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929c12a-fa4a-48d6-b81e-f8bccd4f99c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb41d8ee37f8426d9ac14efbbda441a5)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:55.499030 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929c12a-fa4a-48d6-b81e-f8bccd4f99c5 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:55.548686 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c929c12a-fa4a-48d6-b81e-f8bccd4f99c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb41d8ee37f8426d9ac14efbbda441a5'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.549335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c929c12a-fa4a-48d6-b81e-f8bccd4f99c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb41d8ee37f8426d9ac14efbbda441a5'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.550215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedabfcf-1cea-465e-9116-d253cf35dc61 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:55.554846 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:55.555780 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:55.556064 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:55.556339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] PATH_INFO: `/v3/users/11b17893de284ab2821f033581e2fa64` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:55.557537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b17893de284ab2821f033581e2fa64)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:55.560373 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:55.564006 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c929c12a-fa4a-48d6-b81e-f8bccd4f99c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb41d8ee37f8426d9ac14efbbda441a5'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.574748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c929c12a-fa4a-48d6-b81e-f8bccd4f99c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb41d8ee37f8426d9ac14efbbda441a5'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.575519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c929c12a-fa4a-48d6-b81e-f8bccd4f99c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb41d8ee37f8426d9ac14efbbda441a5'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.593050 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1169/2301] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:55 2022] DELETE /identity/v3/users/bb41d8ee37f8426d9ac14efbbda441a5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:55.598477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bcb2f-76f7-4d5c-8964-13027e1b7132 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:55.598977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bcb2f-76f7-4d5c-8964-13027e1b7132 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:55.599364 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bcb2f-76f7-4d5c-8964-13027e1b7132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:55.628196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b17893de284ab2821f033581e2fa64'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.628741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-553bcb2f-76f7-4d5c-8964-13027e1b7132 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:55.630038 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-553bcb2f-76f7-4d5c-8964-13027e1b7132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:55.630435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b17893de284ab2821f033581e2fa64'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.631083 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b17893de284ab2821f033581e2fa64'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.641480 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b17893de284ab2821f033581e2fa64'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.658587 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b17893de284ab2821f033581e2fa64'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:55.680828 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1133/2302] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:13:55 2022] DELETE /identity/v3/users/11b17893de284ab2821f033581e2fa64 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:55.708526 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1170/2303] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:55.736412 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:55.738146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:55.819774 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:55.825414 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e85b3a-c141-422a-a9ef-129890fe4b2e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:55.831049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a81b5-c013-42ac-ae45-fd314b340fc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:55.832326 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a81b5-c013-42ac-ae45-fd314b340fc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:55.833553 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a81b5-c013-42ac-ae45-fd314b340fc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:55.834006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a81b5-c013-42ac-ae45-fd314b340fc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:55.908846 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a81b5-c013-42ac-ae45-fd314b340fc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:55.910501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a81b5-c013-42ac-ae45-fd314b340fc9 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:55.921086 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1134/2304] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:56.119122 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bcb2f-76f7-4d5c-8964-13027e1b7132 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:56.120817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-553bcb2f-76f7-4d5c-8964-13027e1b7132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:56.143774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bcb2f-76f7-4d5c-8964-13027e1b7132 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:56.166939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:56.168576 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.168992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.169396 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] PATH_INFO: `/v3/projects/8bdee6aaf1b74a53b265fc05bd2c50eb` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.173032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bdee6aaf1b74a53b265fc05bd2c50eb)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:56.177625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:56.275184 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bdee6aaf1b74a53b265fc05bd2c50eb', 'request_id': 'req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:56.281096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bdee6aaf1b74a53b265fc05bd2c50eb', 'request_id': 'req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:56.288013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bdee6aaf1b74a53b265fc05bd2c50eb', 'request_id': 'req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:56.291053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:56.312459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] The token cache is being invalidate because project 8bdee6aaf1b74a53b265fc05bd2c50eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:56.312459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:56.312459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:56.314941 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1171/2305] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:56 2022] DELETE /identity/v3/projects/8bdee6aaf1b74a53b265fc05bd2c50eb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:56.316521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b94af-9f81-4e5c-8684-87ccf6e57ee1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.316815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b94af-9f81-4e5c-8684-87ccf6e57ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.317132 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b94af-9f81-4e5c-8684-87ccf6e57ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.347661 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-da3b94af-9f81-4e5c-8684-87ccf6e57ee1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:56.349366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-da3b94af-9f81-4e5c-8684-87ccf6e57ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:56.372130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:56.374794 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:56.447028 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1135/2306] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:56.456725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a185bb-2060-4b2a-a42e-cc7e6313e5cf None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.467123 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a185bb-2060-4b2a-a42e-cc7e6313e5cf None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.467417 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a185bb-2060-4b2a-a42e-cc7e6313e5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.497522 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-19a185bb-2060-4b2a-a42e-cc7e6313e5cf None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:56.498801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-19a185bb-2060-4b2a-a42e-cc7e6313e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:56.507647 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1136/2307] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:56.513418 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b087dc-0a61-4e31-a9fe-095c0bd03bc9 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:56.517810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a185bb-2060-4b2a-a42e-cc7e6313e5cf None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:56.519375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:56.520533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.520814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.521135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] PATH_INFO: `/v3/projects/1ad13e7306d74bb68b895ec28c1bfd3c` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.523030 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ad13e7306d74bb68b895ec28c1bfd3c)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:56.525617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:56.535142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a185bb-2060-4b2a-a42e-cc7e6313e5cf None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:56.550659 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a07c2-2f94-431a-b26b-a79a1dd85818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:56.551280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a07c2-2f94-431a-b26b-a79a1dd85818 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.551722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a07c2-2f94-431a-b26b-a79a1dd85818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.552037 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a07c2-2f94-431a-b26b-a79a1dd85818 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.552901 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a07c2-2f94-431a-b26b-a79a1dd85818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:56.553319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a07c2-2f94-431a-b26b-a79a1dd85818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:56.555178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a07c2-2f94-431a-b26b-a79a1dd85818 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:56.569773 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1137/2308] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:56.573746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a07c2-2f94-431a-b26b-a79a1dd85818 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:56.587473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a07c2-2f94-431a-b26b-a79a1dd85818 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:56.595338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef2333-8f11-48b8-9aea-aaeceb0b19c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:56.595338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef2333-8f11-48b8-9aea-aaeceb0b19c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.595598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef2333-8f11-48b8-9aea-aaeceb0b19c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.595854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef2333-8f11-48b8-9aea-aaeceb0b19c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.596386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef2333-8f11-48b8-9aea-aaeceb0b19c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:56.602697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef2333-8f11-48b8-9aea-aaeceb0b19c5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:56.630066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ad13e7306d74bb68b895ec28c1bfd3c', 'request_id': 'req-8a40786c-3cac-43ed-bac4-8a947548283a'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:56.643355 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ad13e7306d74bb68b895ec28c1bfd3c', 'request_id': 'req-8a40786c-3cac-43ed-bac4-8a947548283a'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:56.650600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ad13e7306d74bb68b895ec28c1bfd3c', 'request_id': 'req-8a40786c-3cac-43ed-bac4-8a947548283a'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:56.657957 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1138/2309] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:56 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:56.659453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef2333-8f11-48b8-9aea-aaeceb0b19c5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:56.677736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] The token cache is being invalidate because project 1ad13e7306d74bb68b895ec28c1bfd3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:13:56.678041 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:56.678984 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-8a40786c-3cac-43ed-bac4-8a947548283a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:13:56.682366 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1172/2310] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:13:56 2022] DELETE /identity/v3/projects/1ad13e7306d74bb68b895ec28c1bfd3c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:56.687170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ce8c9-7a24-4ce2-9102-c1a9119a491f None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.687710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ce8c9-7a24-4ce2-9102-c1a9119a491f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.688156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ce8c9-7a24-4ce2-9102-c1a9119a491f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.710319 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-c56ce8c9-7a24-4ce2-9102-c1a9119a491f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:56.711722 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c56ce8c9-7a24-4ce2-9102-c1a9119a491f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:56.766682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef2333-8f11-48b8-9aea-aaeceb0b19c5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:56.767647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:56.774802 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84de7f-79e0-4955-8442-c5a99531964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:56.775985 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84de7f-79e0-4955-8442-c5a99531964d admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.776574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84de7f-79e0-4955-8442-c5a99531964d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.777054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84de7f-79e0-4955-8442-c5a99531964d admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.779497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84de7f-79e0-4955-8442-c5a99531964d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:56.782724 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84de7f-79e0-4955-8442-c5a99531964d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:56.815153 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1139/2311] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:56 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:56.817151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84de7f-79e0-4955-8442-c5a99531964d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:56.820882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8a84de7f-79e0-4955-8442-c5a99531964d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:56.829923 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1173/2312] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:56.838037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2830162-343d-40e6-89be-1c7627602bf5 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.838527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2830162-343d-40e6-89be-1c7627602bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.838959 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2830162-343d-40e6-89be-1c7627602bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.847814 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84de7f-79e0-4955-8442-c5a99531964d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:56.861500 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002872c6-8588-44c5-8534-df0efa6da084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:56.862196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002872c6-8588-44c5-8534-df0efa6da084 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.862516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002872c6-8588-44c5-8534-df0efa6da084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.862792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002872c6-8588-44c5-8534-df0efa6da084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.863609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002872c6-8588-44c5-8534-df0efa6da084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:56.865174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002872c6-8588-44c5-8534-df0efa6da084 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:56.865438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a2830162-343d-40e6-89be-1c7627602bf5 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:56.867789 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a2830162-343d-40e6-89be-1c7627602bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:56.882561 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1174/2313] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:56.883695 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2830162-343d-40e6-89be-1c7627602bf5 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:56.898469 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1140/2314] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:56.898469 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002872c6-8588-44c5-8534-df0efa6da084 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:56.915379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2830162-343d-40e6-89be-1c7627602bf5 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:56.928253 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4194e26-13f8-4c91-a1dc-c6a1481b587b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:56.928921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4194e26-13f8-4c91-a1dc-c6a1481b587b admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.929209 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4194e26-13f8-4c91-a1dc-c6a1481b587b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.929487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4194e26-13f8-4c91-a1dc-c6a1481b587b admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.930332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4194e26-13f8-4c91-a1dc-c6a1481b587b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:13:56.930647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4194e26-13f8-4c91-a1dc-c6a1481b587b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:56.932313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4194e26-13f8-4c91-a1dc-c6a1481b587b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:56.933593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002872c6-8588-44c5-8534-df0efa6da084 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:56.940404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcff4c-9afc-4c22-bcd9-135f118b8128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:56.941094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcff4c-9afc-4c22-bcd9-135f118b8128 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:56.941379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcff4c-9afc-4c22-bcd9-135f118b8128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:56.941656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcff4c-9afc-4c22-bcd9-135f118b8128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:56.942439 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcff4c-9afc-4c22-bcd9-135f118b8128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:56.948507 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcff4c-9afc-4c22-bcd9-135f118b8128 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:56.969192 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1175/2315] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:13:56.970145 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4194e26-13f8-4c91-a1dc-c6a1481b587b admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:56.980380 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1141/2316] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:56.981361 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:56.984160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcff4c-9afc-4c22-bcd9-135f118b8128 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:56.987241 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4194e26-13f8-4c91-a1dc-c6a1481b587b admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:57.000983 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcff4c-9afc-4c22-bcd9-135f118b8128 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:57.003003 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ce124-9b14-4ceb-b0ef-880dc4c93f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:57.003700 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ce124-9b14-4ceb-b0ef-880dc4c93f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.003988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ce124-9b14-4ceb-b0ef-880dc4c93f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.004266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ce124-9b14-4ceb-b0ef-880dc4c93f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.004974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ce124-9b14-4ceb-b0ef-880dc4c93f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:57.008750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ce124-9b14-4ceb-b0ef-880dc4c93f9d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:57.014045 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e58e3f-c1b6-44af-9f7f-12c376e7444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:57.015578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e58e3f-c1b6-44af-9f7f-12c376e7444d admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.016030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e58e3f-c1b6-44af-9f7f-12c376e7444d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.017401 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e58e3f-c1b6-44af-9f7f-12c376e7444d admin admin] PATH_INFO: `/v3/projects/f81195056e23489c97e50d14fd3daf4b/users/25bc4e795b294b9f919da0548572901d/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.059168 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e58e3f-c1b6-44af-9f7f-12c376e7444d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81195056e23489c97e50d14fd3daf4b, user_id=25bc4e795b294b9f919da0548572901d, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:57.064427 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1176/2317] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:56 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:57.066446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e58e3f-c1b6-44af-9f7f-12c376e7444d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:57.071039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ce124-9b14-4ceb-b0ef-880dc4c93f9d admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:57.096820 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1142/2318] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:56 2022] PUT /identity/v3/projects/f81195056e23489c97e50d14fd3daf4b/users/25bc4e795b294b9f919da0548572901d/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:57.107143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e32f57-bdde-43d3-ada3-c0137c1b9da5 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.107982 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e32f57-bdde-43d3-ada3-c0137c1b9da5 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.109446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e32f57-bdde-43d3-ada3-c0137c1b9da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.169670 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a3e32f57-bdde-43d3-ada3-c0137c1b9da5 None None] MFA Rules not processed for user `25bc4e795b294b9f919da0548572901d`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:57.172880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a3e32f57-bdde-43d3-ada3-c0137c1b9da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:57.221579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ce124-9b14-4ceb-b0ef-880dc4c93f9d admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:57.329896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6731ee-4177-4386-b3a3-b7b89079bd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:57.330682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6731ee-4177-4386-b3a3-b7b89079bd0c admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.330967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6731ee-4177-4386-b3a3-b7b89079bd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.331257 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6731ee-4177-4386-b3a3-b7b89079bd0c admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.332005 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6731ee-4177-4386-b3a3-b7b89079bd0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:57.335651 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6731ee-4177-4386-b3a3-b7b89079bd0c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:57.350538 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1143/2319] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:57.355397 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e32f57-bdde-43d3-ada3-c0137c1b9da5 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:57.379299 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1177/2320] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:57 2022] POST /identity/v3/users => generated 285 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:57.379426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e32f57-bdde-43d3-ada3-c0137c1b9da5 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:57.384401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6731ee-4177-4386-b3a3-b7b89079bd0c admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:57.394503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368259d6-acec-4775-8449-2c03081c7b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:57.395228 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368259d6-acec-4775-8449-2c03081c7b19 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.395519 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368259d6-acec-4775-8449-2c03081c7b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.395801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368259d6-acec-4775-8449-2c03081c7b19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.396511 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368259d6-acec-4775-8449-2c03081c7b19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:57.399003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368259d6-acec-4775-8449-2c03081c7b19 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:57.424729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6731ee-4177-4386-b3a3-b7b89079bd0c admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:57.442395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb337a-a0d6-4b62-874b-2f9eb4b317e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:57.454668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb337a-a0d6-4b62-874b-2f9eb4b317e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.456301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb337a-a0d6-4b62-874b-2f9eb4b317e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.456903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb337a-a0d6-4b62-874b-2f9eb4b317e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.458314 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1144/2321] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:57 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:57.459300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb337a-a0d6-4b62-874b-2f9eb4b317e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:57.460651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368259d6-acec-4775-8449-2c03081c7b19 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:57.462956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb337a-a0d6-4b62-874b-2f9eb4b317e8 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:57.485736 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1178/2322] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:57.490183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb337a-a0d6-4b62-874b-2f9eb4b317e8 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:57.523224 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb337a-a0d6-4b62-874b-2f9eb4b317e8 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:57.605718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ceacf-5c9b-43c8-af9a-381579549e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:57.606717 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ceacf-5c9b-43c8-af9a-381579549e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.607201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ceacf-5c9b-43c8-af9a-381579549e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.607678 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ceacf-5c9b-43c8-af9a-381579549e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.608534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ceacf-5c9b-43c8-af9a-381579549e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:57.610619 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ceacf-5c9b-43c8-af9a-381579549e6c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:57.635736 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1179/2323] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:57.637584 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ceacf-5c9b-43c8-af9a-381579549e6c admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:57.642072 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368259d6-acec-4775-8449-2c03081c7b19 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:57.657052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954391b2-3b71-411a-9071-3c2f90eba799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:57.658077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954391b2-3b71-411a-9071-3c2f90eba799 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.659036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954391b2-3b71-411a-9071-3c2f90eba799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.659514 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954391b2-3b71-411a-9071-3c2f90eba799 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.660403 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954391b2-3b71-411a-9071-3c2f90eba799 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:57.662840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954391b2-3b71-411a-9071-3c2f90eba799 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:57.671335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:57.691844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ceacf-5c9b-43c8-af9a-381579549e6c admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:57.705945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:57.718739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3699869-3d7b-4301-b8f2-2703c346eb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:57.719432 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1145/2324] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:57 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:13:57.719898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699869-3d7b-4301-b8f2-2703c346eb3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.720190 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699869-3d7b-4301-b8f2-2703c346eb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.720474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699869-3d7b-4301-b8f2-2703c346eb3d admin admin] PATH_INFO: `/v3/projects/815e83f886cc4c78bcdc31eb4230c782/users/9966a96281b745e684d0cd16bb470e6a/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.723414 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954391b2-3b71-411a-9071-3c2f90eba799 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:57.746064 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954391b2-3b71-411a-9071-3c2f90eba799 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:57.759478 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e6353-d343-40f4-ae63-a4cd05adc949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:57.761336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3699869-3d7b-4301-b8f2-2703c346eb3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=815e83f886cc4c78bcdc31eb4230c782, user_id=9966a96281b745e684d0cd16bb470e6a, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:57.769336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6353-d343-40f4-ae63-a4cd05adc949 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.769914 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6353-d343-40f4-ae63-a4cd05adc949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.770344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6353-d343-40f4-ae63-a4cd05adc949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.770819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3699869-3d7b-4301-b8f2-2703c346eb3d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:57.771312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e6353-d343-40f4-ae63-a4cd05adc949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:57.773058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e6353-d343-40f4-ae63-a4cd05adc949 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:57.787598 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1146/2325] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:13:57.792266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e6353-d343-40f4-ae63-a4cd05adc949 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:57.800712 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1180/2326] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:57 2022] PUT /identity/v3/projects/815e83f886cc4c78bcdc31eb4230c782/users/9966a96281b745e684d0cd16bb470e6a/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:57.806962 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbabbb-4c45-44da-aba0-db8f4b62d9fa None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.807401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbabbb-4c45-44da-aba0-db8f4b62d9fa None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.807840 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbabbb-4c45-44da-aba0-db8f4b62d9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.815881 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e6353-d343-40f4-ae63-a4cd05adc949 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:57.840526 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-96dbabbb-4c45-44da-aba0-db8f4b62d9fa None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:57.842929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-96dbabbb-4c45-44da-aba0-db8f4b62d9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:57.898102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3c500-5a1d-4697-9af0-be8a67aa3377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:57.898995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3c500-5a1d-4697-9af0-be8a67aa3377 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.899278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3c500-5a1d-4697-9af0-be8a67aa3377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.899551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3c500-5a1d-4697-9af0-be8a67aa3377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.900356 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3c500-5a1d-4697-9af0-be8a67aa3377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:57.901995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3c500-5a1d-4697-9af0-be8a67aa3377 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:57.923954 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1147/2327] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:57.925409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3c500-5a1d-4697-9af0-be8a67aa3377 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:57.932451 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1181/2328] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:13:57.945707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3c500-5a1d-4697-9af0-be8a67aa3377 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:57.957164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbabbb-4c45-44da-aba0-db8f4b62d9fa None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:57.957164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-96dbabbb-4c45-44da-aba0-db8f4b62d9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:57.957536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae3b47-7c35-4204-8fe1-2b07741446ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:57.957536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae3b47-7c35-4204-8fe1-2b07741446ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:57.957536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae3b47-7c35-4204-8fe1-2b07741446ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:57.957536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae3b47-7c35-4204-8fe1-2b07741446ea admin admin] PATH_INFO: `/v3/projects/c3d5d605c316418caec3aa8b42374559/users/694b84e96b524fdc802e6fd0e7f1d822/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:57.973882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae3b47-7c35-4204-8fe1-2b07741446ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d5d605c316418caec3aa8b42374559, user_id=694b84e96b524fdc802e6fd0e7f1d822, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:57.977160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae3b47-7c35-4204-8fe1-2b07741446ea admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:58.007006 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1148/2329] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:57 2022] PUT /identity/v3/projects/c3d5d605c316418caec3aa8b42374559/users/694b84e96b524fdc802e6fd0e7f1d822/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:58.017925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19010c4-7cd2-41c8-a0d6-9c84341b0a1c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:58.018513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19010c4-7cd2-41c8-a0d6-9c84341b0a1c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:58.019112 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19010c4-7cd2-41c8-a0d6-9c84341b0a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:58.025835 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbabbb-4c45-44da-aba0-db8f4b62d9fa None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:58.054133 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-f19010c4-7cd2-41c8-a0d6-9c84341b0a1c None None] MFA Rules not processed for user `694b84e96b524fdc802e6fd0e7f1d822`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:58.055637 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f19010c4-7cd2-41c8-a0d6-9c84341b0a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:58.088726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a29b72-d2af-4a2f-a98b-56a6b5f991c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:58.088989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a29b72-d2af-4a2f-a98b-56a6b5f991c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:58.089273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a29b72-d2af-4a2f-a98b-56a6b5f991c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:58.089552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a29b72-d2af-4a2f-a98b-56a6b5f991c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:58.137574 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1149/2330] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:58.141887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19010c4-7cd2-41c8-a0d6-9c84341b0a1c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:58.147915 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:58.166984 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a29b72-d2af-4a2f-a98b-56a6b5f991c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:58.168668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a29b72-d2af-4a2f-a98b-56a6b5f991c0 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:58.181226 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1182/2331] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:13:58.201435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:58.219642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19010c4-7cd2-41c8-a0d6-9c84341b0a1c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:58.224366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e6866-4ad4-4796-9233-2941f2459bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:58.225190 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e6866-4ad4-4796-9233-2941f2459bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:58.225845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e6866-4ad4-4796-9233-2941f2459bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:58.225845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e6866-4ad4-4796-9233-2941f2459bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:58.226653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e6866-4ad4-4796-9233-2941f2459bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:58.228689 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e6866-4ad4-4796-9233-2941f2459bdd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:58.262300 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1150/2332] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:13:58 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:58.263093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a29b72-d2af-4a2f-a98b-56a6b5f991c0 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:58.276217 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a29b72-d2af-4a2f-a98b-56a6b5f991c0 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:58.359402 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d2488-409a-422f-b79f-e221aa509021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:58.360926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d2488-409a-422f-b79f-e221aa509021 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:58.361280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d2488-409a-422f-b79f-e221aa509021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:58.361654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d2488-409a-422f-b79f-e221aa509021 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:58.362736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d2488-409a-422f-b79f-e221aa509021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:58.366645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d2488-409a-422f-b79f-e221aa509021 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:58.401712 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1183/2333] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:13:58 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:13:58.406086 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e6866-4ad4-4796-9233-2941f2459bdd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:58.419795 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e6866-4ad4-4796-9233-2941f2459bdd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:58.425582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd6202-e7e6-4008-92ef-4fe0f26d8820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:58.426523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd6202-e7e6-4008-92ef-4fe0f26d8820 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:58.426789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd6202-e7e6-4008-92ef-4fe0f26d8820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:58.427217 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd6202-e7e6-4008-92ef-4fe0f26d8820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:58.428000 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd6202-e7e6-4008-92ef-4fe0f26d8820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:58.429672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd6202-e7e6-4008-92ef-4fe0f26d8820 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:58.440799 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1151/2334] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:58.444973 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d2488-409a-422f-b79f-e221aa509021 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:58.455409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d2488-409a-422f-b79f-e221aa509021 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:58.460406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823abed-a4e3-4337-a838-47d576cb81cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:58.461503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823abed-a4e3-4337-a838-47d576cb81cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:58.461788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823abed-a4e3-4337-a838-47d576cb81cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:58.462062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823abed-a4e3-4337-a838-47d576cb81cc admin admin] PATH_INFO: `/v3/projects/1a2cd0982b6a435cac7b0568c77cf92a/users/a5db0f0667a2412db411a0ede5ff1a1c/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:58.477354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823abed-a4e3-4337-a838-47d576cb81cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2cd0982b6a435cac7b0568c77cf92a, user_id=a5db0f0667a2412db411a0ede5ff1a1c, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:58.480054 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823abed-a4e3-4337-a838-47d576cb81cc admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:58.494957 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1184/2335] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:58 2022] PUT /identity/v3/projects/1a2cd0982b6a435cac7b0568c77cf92a/users/a5db0f0667a2412db411a0ede5ff1a1c/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:13:58.499159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd6202-e7e6-4008-92ef-4fe0f26d8820 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:58.509722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd6202-e7e6-4008-92ef-4fe0f26d8820 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:58.586108 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e35ee9-64b1-41c1-9185-45e663825eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:58.586909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e35ee9-64b1-41c1-9185-45e663825eba admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:58.587249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e35ee9-64b1-41c1-9185-45e663825eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:58.587535 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e35ee9-64b1-41c1-9185-45e663825eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:58.588328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e35ee9-64b1-41c1-9185-45e663825eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:58.589951 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e35ee9-64b1-41c1-9185-45e663825eba admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:58.600710 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1152/2336] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:13:58.604827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823abed-a4e3-4337-a838-47d576cb81cc admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:58.626045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823abed-a4e3-4337-a838-47d576cb81cc admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:58.632321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0f836-3020-40d9-9ef5-2b9bc27d3689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:58.633527 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0f836-3020-40d9-9ef5-2b9bc27d3689 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:58.634086 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0f836-3020-40d9-9ef5-2b9bc27d3689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:58.634605 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0f836-3020-40d9-9ef5-2b9bc27d3689 admin admin] PATH_INFO: `/v3/projects/1a2cd0982b6a435cac7b0568c77cf92a/users/a5db0f0667a2412db411a0ede5ff1a1c/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:58.637000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0f836-3020-40d9-9ef5-2b9bc27d3689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2cd0982b6a435cac7b0568c77cf92a, user_id=a5db0f0667a2412db411a0ede5ff1a1c, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:58.639766 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0f836-3020-40d9-9ef5-2b9bc27d3689 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:58.652981 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1185/2337] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:13:58 2022] PUT /identity/v3/projects/1a2cd0982b6a435cac7b0568c77cf92a/users/a5db0f0667a2412db411a0ede5ff1a1c/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:13:58.722891 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b5952-ef2d-4044-bbdf-42f41884e914 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:58.723431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b5952-ef2d-4044-bbdf-42f41884e914 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:58.723431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b5952-ef2d-4044-bbdf-42f41884e914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:58.750754 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c03b5952-ef2d-4044-bbdf-42f41884e914 None None] MFA Rules not processed for user `a5db0f0667a2412db411a0ede5ff1a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:13:58.752043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c03b5952-ef2d-4044-bbdf-42f41884e914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:58.847351 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1153/2338] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:13:58.859134 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0f836-3020-40d9-9ef5-2b9bc27d3689 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:13:58.861078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-69f0f836-3020-40d9-9ef5-2b9bc27d3689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:13:58.956118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0f836-3020-40d9-9ef5-2b9bc27d3689 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:13:58.963256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641859d7-d65f-4f8b-92db-213b0889815b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:13:58.963935 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641859d7-d65f-4f8b-92db-213b0889815b service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:13:58.964260 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641859d7-d65f-4f8b-92db-213b0889815b service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:13:58.964544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641859d7-d65f-4f8b-92db-213b0889815b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:13:59.051127 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:59.055916 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641859d7-d65f-4f8b-92db-213b0889815b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:13:59.058175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641859d7-d65f-4f8b-92db-213b0889815b service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:13:59.077329 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1186/2339] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:13:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:02.143961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b5952-ef2d-4044-bbdf-42f41884e914 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:02.160391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b5952-ef2d-4044-bbdf-42f41884e914 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:02.186418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2cb2c-2d7e-4a0f-9c24-2d67d97749ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:02.187311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2cb2c-2d7e-4a0f-9c24-2d67d97749ad service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:02.187624 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2cb2c-2d7e-4a0f-9c24-2d67d97749ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:02.187955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2cb2c-2d7e-4a0f-9c24-2d67d97749ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:02.293157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:02.326456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2cb2c-2d7e-4a0f-9c24-2d67d97749ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:02.328978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2cb2c-2d7e-4a0f-9c24-2d67d97749ad service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:02.348683 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1154/2340] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:02.481033 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbecffa-d7f7-488e-90b7-68e1d4c28c48 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:02.481591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbecffa-d7f7-488e-90b7-68e1d4c28c48 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:02.481591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbecffa-d7f7-488e-90b7-68e1d4c28c48 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:02.483007 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1187/2341] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:14:02.505186 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494936-c23c-41c2-a616-858ce5235535 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:02.505478 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494936-c23c-41c2-a616-858ce5235535 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:02.505762 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d494936-c23c-41c2-a616-858ce5235535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:02.535630 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3d494936-c23c-41c2-a616-858ce5235535 None None] MFA Rules not processed for user `e9d4a50f842a48b1875900426c6a1ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:02.537307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3d494936-c23c-41c2-a616-858ce5235535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:02.653744 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbecffa-d7f7-488e-90b7-68e1d4c28c48 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:02.654270 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1155/2342] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:14:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:02.656068 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4bbecffa-d7f7-488e-90b7-68e1d4c28c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:02.687293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d494936-c23c-41c2-a616-858ce5235535 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:02.689027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3d494936-c23c-41c2-a616-858ce5235535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:02.712102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d494936-c23c-41c2-a616-858ce5235535 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:02.720993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bddff-aedd-4f77-8d53-effdd3041995 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:02.722513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bddff-aedd-4f77-8d53-effdd3041995 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:02.723121 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bddff-aedd-4f77-8d53-effdd3041995 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:02.723751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bddff-aedd-4f77-8d53-effdd3041995 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:02.738574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bddff-aedd-4f77-8d53-effdd3041995 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:02.741006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bddff-aedd-4f77-8d53-effdd3041995 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:02.784790 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1156/2343] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:02.829752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbecffa-d7f7-488e-90b7-68e1d4c28c48 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:02.841354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:02.842458 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:02.842742 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:02.843016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] PATH_INFO: `/v3/users/c323490c11b9461898687e729cd357f2` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:02.845088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c323490c11b9461898687e729cd357f2)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:02.847094 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:02.921432 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c323490c11b9461898687e729cd357f2'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:02.922924 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c323490c11b9461898687e729cd357f2'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:02.940951 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c323490c11b9461898687e729cd357f2'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:02.950058 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c323490c11b9461898687e729cd357f2'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:02.950906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c323490c11b9461898687e729cd357f2'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:02.986120 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1188/2344] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:02 2022] DELETE /identity/v3/users/c323490c11b9461898687e729cd357f2 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:03.061205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bddff-aedd-4f77-8d53-effdd3041995 service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:03.064311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-798bddff-aedd-4f77-8d53-effdd3041995 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:03.093030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bddff-aedd-4f77-8d53-effdd3041995 service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:03.210152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb52f8-4dcc-48f6-a817-f31187b06dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:03.211299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb52f8-4dcc-48f6-a817-f31187b06dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:03.211584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb52f8-4dcc-48f6-a817-f31187b06dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:03.212246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb52f8-4dcc-48f6-a817-f31187b06dbd admin admin] PATH_INFO: `/v3/users/8603d5ce5ab840f7948fd111108e65ac` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:03.215630 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb52f8-4dcc-48f6-a817-f31187b06dbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8603d5ce5ab840f7948fd111108e65ac)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:03.221108 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb52f8-4dcc-48f6-a817-f31187b06dbd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:03.312707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-68fb52f8-4dcc-48f6-a817-f31187b06dbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8603d5ce5ab840f7948fd111108e65ac'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:03.313580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-68fb52f8-4dcc-48f6-a817-f31187b06dbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8603d5ce5ab840f7948fd111108e65ac'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:03.315657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-68fb52f8-4dcc-48f6-a817-f31187b06dbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8603d5ce5ab840f7948fd111108e65ac'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:03.324359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-68fb52f8-4dcc-48f6-a817-f31187b06dbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8603d5ce5ab840f7948fd111108e65ac'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:03.328255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:03.330083 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:03.348433 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:03.357253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-68fb52f8-4dcc-48f6-a817-f31187b06dbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8603d5ce5ab840f7948fd111108e65ac'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:03.377497 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1157/2345] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:03 2022] DELETE /identity/v3/users/8603d5ce5ab840f7948fd111108e65ac => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:03.381825 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079dbfd-9f99-46ec-a89d-75138e7fddd1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:03.382284 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079dbfd-9f99-46ec-a89d-75138e7fddd1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:03.382771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079dbfd-9f99-46ec-a89d-75138e7fddd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:03.403530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1079dbfd-9f99-46ec-a89d-75138e7fddd1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:03.404487 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1079dbfd-9f99-46ec-a89d-75138e7fddd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:03.447516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7755a91-aa77-4fc9-8047-ddedab4f12d5 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:03.485693 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1158/2346] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:03.496709 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079dbfd-9f99-46ec-a89d-75138e7fddd1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:03.500617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1079dbfd-9f99-46ec-a89d-75138e7fddd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:03.507372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:03.524568 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079dbfd-9f99-46ec-a89d-75138e7fddd1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:03.529722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:03.531203 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:03.531494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:03.531785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] PATH_INFO: `/v3/projects/4a7e4c4e4d154bd9a7197190a9d8b27f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:03.533065 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a7e4c4e4d154bd9a7197190a9d8b27f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:03.538260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:03.548706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8c9b9-3529-4c8a-ac6a-daf4527158b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:03.549378 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8c9b9-3529-4c8a-ac6a-daf4527158b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:03.549657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8c9b9-3529-4c8a-ac6a-daf4527158b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:03.549928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8c9b9-3529-4c8a-ac6a-daf4527158b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:03.558707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4ab8c9b9-3529-4c8a-ac6a-daf4527158b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:03.645967 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a7e4c4e4d154bd9a7197190a9d8b27f', 'request_id': 'req-74c83e5a-cc96-4991-9aa8-510061303127'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:03.662950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a7e4c4e4d154bd9a7197190a9d8b27f', 'request_id': 'req-74c83e5a-cc96-4991-9aa8-510061303127'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:03.674413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a7e4c4e4d154bd9a7197190a9d8b27f', 'request_id': 'req-74c83e5a-cc96-4991-9aa8-510061303127'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:03.705709 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] The token cache is being invalidate because project 4a7e4c4e4d154bd9a7197190a9d8b27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:03.706740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:03.708013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:03.713925 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1159/2347] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:03 2022] DELETE /identity/v3/projects/4a7e4c4e4d154bd9a7197190a9d8b27f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:03.713925 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:03.719386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:03.750827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8c9b9-3529-4c8a-ac6a-daf4527158b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:03.752357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8c9b9-3529-4c8a-ac6a-daf4527158b0 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:03.780298 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1189/2348] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:03.780987 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8c9b9-3529-4c8a-ac6a-daf4527158b0 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:03.792235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8c9b9-3529-4c8a-ac6a-daf4527158b0 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:03.852126 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c83e5a-cc96-4991-9aa8-510061303127 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:03.862635 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991c74b-264d-4bfb-91ef-3b65bed42f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:03.863441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991c74b-264d-4bfb-91ef-3b65bed42f8c service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:03.863727 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991c74b-264d-4bfb-91ef-3b65bed42f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:03.864008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991c74b-264d-4bfb-91ef-3b65bed42f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:03.868093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b991c74b-264d-4bfb-91ef-3b65bed42f8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:03.877482 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:03.878668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:03.879044 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:03.879271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] PATH_INFO: `/v3/users/1c639857118347fea2e5927efaaa492f` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:03.880751 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c639857118347fea2e5927efaaa492f)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:03.894527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991c74b-264d-4bfb-91ef-3b65bed42f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:03.896737 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991c74b-264d-4bfb-91ef-3b65bed42f8c service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:03.903359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:03.919163 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1160/2349] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:04.058351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c639857118347fea2e5927efaaa492f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.076625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c639857118347fea2e5927efaaa492f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.143166 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c639857118347fea2e5927efaaa492f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.160174 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c639857118347fea2e5927efaaa492f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.161853 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c639857118347fea2e5927efaaa492f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.178076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:04.186051 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1190/2350] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:03 2022] DELETE /identity/v3/users/1c639857118347fea2e5927efaaa492f => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:04.463938 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991c74b-264d-4bfb-91ef-3b65bed42f8c service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:04.465689 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b991c74b-264d-4bfb-91ef-3b65bed42f8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:04.479679 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991c74b-264d-4bfb-91ef-3b65bed42f8c service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:04.498097 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:04.536702 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:04.538159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:04.572293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:04.573638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:04.573999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:04.574415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] PATH_INFO: `/v3/projects/07e46e591bba42b1acaf36faface1f0c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:04.575791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07e46e591bba42b1acaf36faface1f0c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:04.577900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:04.650162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab7851-0ded-4452-a7e2-22456cc362b2 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:04.653847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07e46e591bba42b1acaf36faface1f0c', 'request_id': 'req-aba326bd-4700-4f28-b11a-13348dded577'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.674262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07e46e591bba42b1acaf36faface1f0c', 'request_id': 'req-aba326bd-4700-4f28-b11a-13348dded577'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.683758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07e46e591bba42b1acaf36faface1f0c', 'request_id': 'req-aba326bd-4700-4f28-b11a-13348dded577'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.697808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] The token cache is being invalidate because project 07e46e591bba42b1acaf36faface1f0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:04.699295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.700004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-aba326bd-4700-4f28-b11a-13348dded577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.714246 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1161/2351] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:04 2022] DELETE /identity/v3/projects/07e46e591bba42b1acaf36faface1f0c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:04.772496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:04.774043 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:04.774455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:04.774830 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] PATH_INFO: `/v3/projects/fd9c63344ebc46eead81ad9d7269e239` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:04.792718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9c63344ebc46eead81ad9d7269e239)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:04.794794 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:04.902105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9c63344ebc46eead81ad9d7269e239', 'request_id': 'req-d1185103-2364-4f30-96a4-5d5180bb76c9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.915125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9c63344ebc46eead81ad9d7269e239', 'request_id': 'req-d1185103-2364-4f30-96a4-5d5180bb76c9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.925592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9c63344ebc46eead81ad9d7269e239', 'request_id': 'req-d1185103-2364-4f30-96a4-5d5180bb76c9'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.948510 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] The token cache is being invalidate because project fd9c63344ebc46eead81ad9d7269e239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:04.949210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.950064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-d1185103-2364-4f30-96a4-5d5180bb76c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:04.951990 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1191/2352] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:04 2022] DELETE /identity/v3/projects/fd9c63344ebc46eead81ad9d7269e239 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:14:04.957840 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bec92-e424-4b6c-be97-3f070ed5105a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:04.958201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bec92-e424-4b6c-be97-3f070ed5105a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:04.958637 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bec92-e424-4b6c-be97-3f070ed5105a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:04.993936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a46bec92-e424-4b6c-be97-3f070ed5105a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:04.995172 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a46bec92-e424-4b6c-be97-3f070ed5105a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:05.130880 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1162/2353] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:05.138992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea41848-dcfc-4a57-a125-711950886a8b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.139393 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea41848-dcfc-4a57-a125-711950886a8b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.139735 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea41848-dcfc-4a57-a125-711950886a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:05.174264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-4ea41848-dcfc-4a57-a125-711950886a8b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:05.176046 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4ea41848-dcfc-4a57-a125-711950886a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:05.201041 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1192/2354] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:05.205549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bec92-e424-4b6c-be97-3f070ed5105a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:05.221538 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bec92-e424-4b6c-be97-3f070ed5105a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:05.238156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc83b6e-a751-4fa8-a58c-13edfad8a4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:05.241672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc83b6e-a751-4fa8-a58c-13edfad8a4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.242150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc83b6e-a751-4fa8-a58c-13edfad8a4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.242650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc83b6e-a751-4fa8-a58c-13edfad8a4f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:05.244694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc83b6e-a751-4fa8-a58c-13edfad8a4f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:14:05.248214 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc83b6e-a751-4fa8-a58c-13edfad8a4f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:05.255254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc83b6e-a751-4fa8-a58c-13edfad8a4f9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:05.294772 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea41848-dcfc-4a57-a125-711950886a8b None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:05.297384 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1163/2355] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:14:05.304294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea41848-dcfc-4a57-a125-711950886a8b None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:05.325768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568df52-7fbe-4a1e-822b-20eb02dce65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:05.327391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568df52-7fbe-4a1e-822b-20eb02dce65a admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.327830 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568df52-7fbe-4a1e-822b-20eb02dce65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.328252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568df52-7fbe-4a1e-822b-20eb02dce65a admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:05.329481 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568df52-7fbe-4a1e-822b-20eb02dce65a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:05.333810 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568df52-7fbe-4a1e-822b-20eb02dce65a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:05.389677 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1193/2356] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:14:05 2022] POST /identity/v3/projects => generated 359 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:05.391122 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc83b6e-a751-4fa8-a58c-13edfad8a4f9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:05.404548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc83b6e-a751-4fa8-a58c-13edfad8a4f9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:05.451480 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:05.482851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ada49-1243-4f42-8c71-8efac121f561 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.483141 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ada49-1243-4f42-8c71-8efac121f561 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.483464 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ada49-1243-4f42-8c71-8efac121f561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:05.502559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:05.553109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a07ada49-1243-4f42-8c71-8efac121f561 None None] MFA Rules not processed for user `9966a96281b745e684d0cd16bb470e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:05.554376 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a07ada49-1243-4f42-8c71-8efac121f561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:05.579875 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6936f-00ac-4800-8dc5-b0b4d54fa9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:05.581205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6936f-00ac-4800-8dc5-b0b4d54fa9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.581643 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6936f-00ac-4800-8dc5-b0b4d54fa9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.582178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6936f-00ac-4800-8dc5-b0b4d54fa9c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:05.584936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c6936f-00ac-4800-8dc5-b0b4d54fa9c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:05.590703 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c6936f-00ac-4800-8dc5-b0b4d54fa9c2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:05.653526 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1164/2357] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:14:05 2022] POST /identity/v3/users => generated 280 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:05.654538 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6936f-00ac-4800-8dc5-b0b4d54fa9c2 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:05.678900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6936f-00ac-4800-8dc5-b0b4d54fa9c2 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:05.692532 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1194/2358] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:05.697189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ada49-1243-4f42-8c71-8efac121f561 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:05.699086 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef365cf-508a-418f-8ba9-c69895e677a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:05.699539 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a07ada49-1243-4f42-8c71-8efac121f561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:05.700215 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef365cf-508a-418f-8ba9-c69895e677a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.700680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef365cf-508a-418f-8ba9-c69895e677a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.702769 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef365cf-508a-418f-8ba9-c69895e677a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:05.703970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef365cf-508a-418f-8ba9-c69895e677a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:05.708855 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef365cf-508a-418f-8ba9-c69895e677a8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:05.719244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ada49-1243-4f42-8c71-8efac121f561 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:05.725552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913cd96-7382-439a-92f6-ce90859ef232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:05.729556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cd96-7382-439a-92f6-ce90859ef232 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.729911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cd96-7382-439a-92f6-ce90859ef232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.730197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cd96-7382-439a-92f6-ce90859ef232 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:05.733181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913cd96-7382-439a-92f6-ce90859ef232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:05.735180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913cd96-7382-439a-92f6-ce90859ef232 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:05.767344 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1165/2359] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:05.775569 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef365cf-508a-418f-8ba9-c69895e677a8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:05.790394 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1195/2360] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:14:05 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:14:05.794382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913cd96-7382-439a-92f6-ce90859ef232 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:05.802803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef365cf-508a-418f-8ba9-c69895e677a8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:05.886429 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913cd96-7382-439a-92f6-ce90859ef232 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:05.892471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530a043-eb8d-4ccf-b9ff-e0143104341d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:05.893094 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a043-eb8d-4ccf-b9ff-e0143104341d admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.893397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a043-eb8d-4ccf-b9ff-e0143104341d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.893654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a043-eb8d-4ccf-b9ff-e0143104341d admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:05.894669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530a043-eb8d-4ccf-b9ff-e0143104341d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:05.896614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530a043-eb8d-4ccf-b9ff-e0143104341d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:05.930945 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1196/2361] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:14:05 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:05.935396 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530a043-eb8d-4ccf-b9ff-e0143104341d admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:05.942956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30779de4-19ea-49d6-bf10-48eed4f0aa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:05.944688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779de4-19ea-49d6-bf10-48eed4f0aa86 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.945354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779de4-19ea-49d6-bf10-48eed4f0aa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.946147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779de4-19ea-49d6-bf10-48eed4f0aa86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:05.947260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30779de4-19ea-49d6-bf10-48eed4f0aa86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:05.948090 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530a043-eb8d-4ccf-b9ff-e0143104341d admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:05.950143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30779de4-19ea-49d6-bf10-48eed4f0aa86 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:05.954617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29bbc2-5c3d-4688-ab76-3ba24dbc3eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:05.955310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29bbc2-5c3d-4688-ab76-3ba24dbc3eed admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.955736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29bbc2-5c3d-4688-ab76-3ba24dbc3eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.956030 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29bbc2-5c3d-4688-ab76-3ba24dbc3eed admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:05.956795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29bbc2-5c3d-4688-ab76-3ba24dbc3eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:05.958969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29bbc2-5c3d-4688-ab76-3ba24dbc3eed admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:05.966377 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1166/2362] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:05.968027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30779de4-19ea-49d6-bf10-48eed4f0aa86 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:05.970261 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1197/2363] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:14:05.973937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29bbc2-5c3d-4688-ab76-3ba24dbc3eed admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:05.982489 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30779de4-19ea-49d6-bf10-48eed4f0aa86 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:05.984871 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29bbc2-5c3d-4688-ab76-3ba24dbc3eed admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:05.988732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42e726-9898-4444-9a28-da9f6442354e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:05.989661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34c839-6dab-46de-bf01-beb472b9a632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:05.989841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42e726-9898-4444-9a28-da9f6442354e admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.990119 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42e726-9898-4444-9a28-da9f6442354e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.990419 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42e726-9898-4444-9a28-da9f6442354e admin admin] PATH_INFO: `/v3/projects/cfbfb0975249469c880b306d0825e599/users/b3f4bf397dc4439a8bc1f4b3f27ac700/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:05.990640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34c839-6dab-46de-bf01-beb472b9a632 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:05.990923 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34c839-6dab-46de-bf01-beb472b9a632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:05.991237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34c839-6dab-46de-bf01-beb472b9a632 admin admin] PATH_INFO: `/v3/projects/cd55aaf5ed51497bb395c22072a4bbd9/users/721521f50380490ea84fb46734f0f592/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:06.010483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42e726-9898-4444-9a28-da9f6442354e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfbfb0975249469c880b306d0825e599, user_id=b3f4bf397dc4439a8bc1f4b3f27ac700, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:06.012124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34c839-6dab-46de-bf01-beb472b9a632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd55aaf5ed51497bb395c22072a4bbd9, user_id=721521f50380490ea84fb46734f0f592, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:06.013258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42e726-9898-4444-9a28-da9f6442354e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:06.014848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34c839-6dab-46de-bf01-beb472b9a632 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:06.030080 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1167/2364] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:14:05 2022] PUT /identity/v3/projects/cfbfb0975249469c880b306d0825e599/users/b3f4bf397dc4439a8bc1f4b3f27ac700/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:06.038499 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1198/2365] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:14:05 2022] PUT /identity/v3/projects/cd55aaf5ed51497bb395c22072a4bbd9/users/721521f50380490ea84fb46734f0f592/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:14:06.038601 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42e726-9898-4444-9a28-da9f6442354e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:06.040635 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91208c-8287-4c8a-972f-5e401457cd80 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:06.042468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91208c-8287-4c8a-972f-5e401457cd80 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:06.042926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91208c-8287-4c8a-972f-5e401457cd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:06.076730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-1b91208c-8287-4c8a-972f-5e401457cd80 None None] MFA Rules not processed for user `b3f4bf397dc4439a8bc1f4b3f27ac700`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:06.078387 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1b91208c-8287-4c8a-972f-5e401457cd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:06.101325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:06.122020 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42e726-9898-4444-9a28-da9f6442354e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:06.156942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:06.162960 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1199/2366] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:06 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:06.199014 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91208c-8287-4c8a-972f-5e401457cd80 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:06.199495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d871a3-97de-4f1e-9ed1-3a8f9ebbe089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:06.200598 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1b91208c-8287-4c8a-972f-5e401457cd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:06.200711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d871a3-97de-4f1e-9ed1-3a8f9ebbe089 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:06.200957 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d871a3-97de-4f1e-9ed1-3a8f9ebbe089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:06.201239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d871a3-97de-4f1e-9ed1-3a8f9ebbe089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:06.202098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d871a3-97de-4f1e-9ed1-3a8f9ebbe089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:06.204141 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d871a3-97de-4f1e-9ed1-3a8f9ebbe089 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:06.222858 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1168/2367] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:14:06.224909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d871a3-97de-4f1e-9ed1-3a8f9ebbe089 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:06.240492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d871a3-97de-4f1e-9ed1-3a8f9ebbe089 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:06.246061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71ec35-fb25-4949-9379-0f996f218d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:06.247160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ec35-fb25-4949-9379-0f996f218d5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:06.247538 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ec35-fb25-4949-9379-0f996f218d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:06.247941 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ec35-fb25-4949-9379-0f996f218d5c admin admin] PATH_INFO: `/v3/projects/cd55aaf5ed51497bb395c22072a4bbd9/users/721521f50380490ea84fb46734f0f592/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:06.250178 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71ec35-fb25-4949-9379-0f996f218d5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd55aaf5ed51497bb395c22072a4bbd9, user_id=721521f50380490ea84fb46734f0f592, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:06.253029 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71ec35-fb25-4949-9379-0f996f218d5c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:06.266466 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1169/2368] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:14:06 2022] PUT /identity/v3/projects/cd55aaf5ed51497bb395c22072a4bbd9/users/721521f50380490ea84fb46734f0f592/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:06.266667 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91208c-8287-4c8a-972f-5e401457cd80 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:06.321202 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b8d8f-9f7a-4cdf-9763-17136f3958dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:06.321946 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b8d8f-9f7a-4cdf-9763-17136f3958dc service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:06.322345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b8d8f-9f7a-4cdf-9763-17136f3958dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:06.322678 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b8d8f-9f7a-4cdf-9763-17136f3958dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:06.387056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b8d8f-9f7a-4cdf-9763-17136f3958dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:06.388742 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b8d8f-9f7a-4cdf-9763-17136f3958dc service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:06.399646 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1200/2369] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:06 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:14:06.414841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71ec35-fb25-4949-9379-0f996f218d5c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:06.531944 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71ec35-fb25-4949-9379-0f996f218d5c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:06.539562 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:06.542426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:06.542895 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:06.543431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] PATH_INFO: `/v3/users/b3f4bf397dc4439a8bc1f4b3f27ac700` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:06.544975 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f4bf397dc4439a8bc1f4b3f27ac700)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:06.547320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:06.647699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f4bf397dc4439a8bc1f4b3f27ac700'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:06.648976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f4bf397dc4439a8bc1f4b3f27ac700'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:06.651160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f4bf397dc4439a8bc1f4b3f27ac700'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:06.671175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f4bf397dc4439a8bc1f4b3f27ac700'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:06.709803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f4bf397dc4439a8bc1f4b3f27ac700'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:06.745570 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1170/2370] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:06 2022] DELETE /identity/v3/users/b3f4bf397dc4439a8bc1f4b3f27ac700 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:06.750563 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4648f0-d902-46e0-bff0-3a1e2b3c451f None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:06.751455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4648f0-d902-46e0-bff0-3a1e2b3c451f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:06.753248 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4648f0-d902-46e0-bff0-3a1e2b3c451f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:06.783321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-9a4648f0-d902-46e0-bff0-3a1e2b3c451f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:06.784556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9a4648f0-d902-46e0-bff0-3a1e2b3c451f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:06.818902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:06.875656 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1201/2371] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:06.885265 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:06.886717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:06.935106 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:06.999614 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8834479-2984-4d3a-a545-3dc987b3a609 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:07.003527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd35078-fc7d-4d4e-976e-cd289099df70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:07.004193 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd35078-fc7d-4d4e-976e-cd289099df70 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:07.004482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd35078-fc7d-4d4e-976e-cd289099df70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:07.004755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd35078-fc7d-4d4e-976e-cd289099df70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:07.025764 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd35078-fc7d-4d4e-976e-cd289099df70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:07.027410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd35078-fc7d-4d4e-976e-cd289099df70 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:07.044175 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1171/2372] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:07.725349 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4648f0-d902-46e0-bff0-3a1e2b3c451f None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:07.726871 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9a4648f0-d902-46e0-bff0-3a1e2b3c451f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:07.772621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4648f0-d902-46e0-bff0-3a1e2b3c451f None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:07.792519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:07.793481 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:07.793766 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:07.794045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] PATH_INFO: `/v3/projects/cfbfb0975249469c880b306d0825e599` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:07.797480 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfbfb0975249469c880b306d0825e599)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:07.799815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:07.898533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfbfb0975249469c880b306d0825e599', 'request_id': 'req-66d475a9-8020-4b17-9955-19660346fd11'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:07.906243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfbfb0975249469c880b306d0825e599', 'request_id': 'req-66d475a9-8020-4b17-9955-19660346fd11'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:07.914331 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfbfb0975249469c880b306d0825e599', 'request_id': 'req-66d475a9-8020-4b17-9955-19660346fd11'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:07.932393 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] The token cache is being invalidate because project cfbfb0975249469c880b306d0825e599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:07.932937 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:07.934822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-66d475a9-8020-4b17-9955-19660346fd11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:07.946435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe78ec-8759-408a-9c73-e07f509fba93 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:07.947798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe78ec-8759-408a-9c73-e07f509fba93 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:07.948316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe78ec-8759-408a-9c73-e07f509fba93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:07.950877 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1202/2373] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:07 2022] DELETE /identity/v3/projects/cfbfb0975249469c880b306d0825e599 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:07.984476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-28fe78ec-8759-408a-9c73-e07f509fba93 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:07.985811 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-28fe78ec-8759-408a-9c73-e07f509fba93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:08.111168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f350ba-98af-4608-adbb-857a6d9b55bf None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:08.111489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f350ba-98af-4608-adbb-857a6d9b55bf None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:08.112540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f350ba-98af-4608-adbb-857a6d9b55bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:08.116383 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1172/2374] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:08.150198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-96f350ba-98af-4608-adbb-857a6d9b55bf None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:08.151817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-96f350ba-98af-4608-adbb-857a6d9b55bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:08.167413 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1203/2375] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:08.171688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe78ec-8759-408a-9c73-e07f509fba93 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:08.185388 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe78ec-8759-408a-9c73-e07f509fba93 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:08.212604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb32c8-3a2d-49aa-a4ff-cfaac5f7f1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:08.213606 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb32c8-3a2d-49aa-a4ff-cfaac5f7f1df admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:08.215259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb32c8-3a2d-49aa-a4ff-cfaac5f7f1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:08.215720 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb32c8-3a2d-49aa-a4ff-cfaac5f7f1df admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:08.219434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb32c8-3a2d-49aa-a4ff-cfaac5f7f1df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:14:08.224235 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb32c8-3a2d-49aa-a4ff-cfaac5f7f1df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:08.233078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb32c8-3a2d-49aa-a4ff-cfaac5f7f1df admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:08.267574 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1173/2376] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:14:08.272330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f350ba-98af-4608-adbb-857a6d9b55bf None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:08.283908 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f350ba-98af-4608-adbb-857a6d9b55bf None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:08.290966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960943b-f775-4bd0-bb20-a40b72f86970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:08.291904 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960943b-f775-4bd0-bb20-a40b72f86970 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:08.292305 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960943b-f775-4bd0-bb20-a40b72f86970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:08.293095 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960943b-f775-4bd0-bb20-a40b72f86970 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:08.294069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960943b-f775-4bd0-bb20-a40b72f86970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:08.298069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960943b-f775-4bd0-bb20-a40b72f86970 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:08.343758 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1204/2377] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:14:08 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:08.345259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb32c8-3a2d-49aa-a4ff-cfaac5f7f1df admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:08.356939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb32c8-3a2d-49aa-a4ff-cfaac5f7f1df admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:08.462815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960943b-f775-4bd0-bb20-a40b72f86970 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:08.465258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3960943b-f775-4bd0-bb20-a40b72f86970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:08.479572 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a9bb1-efb9-4e74-8c0f-b8abe808a9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:08.480526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a9bb1-efb9-4e74-8c0f-b8abe808a9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:08.480822 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a9bb1-efb9-4e74-8c0f-b8abe808a9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:08.481386 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a9bb1-efb9-4e74-8c0f-b8abe808a9f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:08.482378 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a9bb1-efb9-4e74-8c0f-b8abe808a9f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:08.484853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a9bb1-efb9-4e74-8c0f-b8abe808a9f1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:08.524027 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1174/2378] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:14:08 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:08.525329 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a9bb1-efb9-4e74-8c0f-b8abe808a9f1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:08.548947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a9bb1-efb9-4e74-8c0f-b8abe808a9f1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:08.557631 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560b8e3-a960-47dd-8d4c-c54f2ae7dcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:08.558357 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560b8e3-a960-47dd-8d4c-c54f2ae7dcfc admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:08.558647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560b8e3-a960-47dd-8d4c-c54f2ae7dcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:08.558935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560b8e3-a960-47dd-8d4c-c54f2ae7dcfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:08.559685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560b8e3-a960-47dd-8d4c-c54f2ae7dcfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:08.561567 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560b8e3-a960-47dd-8d4c-c54f2ae7dcfc admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:08.586457 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1175/2379] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:08.589998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560b8e3-a960-47dd-8d4c-c54f2ae7dcfc admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:08.595349 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960943b-f775-4bd0-bb20-a40b72f86970 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:08.605386 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d709b-ab1a-4c4b-92c6-7df9ef1e4680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:08.607344 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d709b-ab1a-4c4b-92c6-7df9ef1e4680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:08.607827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d709b-ab1a-4c4b-92c6-7df9ef1e4680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:08.608310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d709b-ab1a-4c4b-92c6-7df9ef1e4680 admin admin] PATH_INFO: `/v3/users/25bc4e795b294b9f919da0548572901d` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:08.610119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d709b-ab1a-4c4b-92c6-7df9ef1e4680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25bc4e795b294b9f919da0548572901d)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:08.612639 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d709b-ab1a-4c4b-92c6-7df9ef1e4680 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:08.616183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560b8e3-a960-47dd-8d4c-c54f2ae7dcfc admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:08.641081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a40a25-1abc-44a0-89b9-1574fb7c274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:08.642014 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a40a25-1abc-44a0-89b9-1574fb7c274e admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:08.642539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a40a25-1abc-44a0-89b9-1574fb7c274e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:08.643464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a40a25-1abc-44a0-89b9-1574fb7c274e admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:08.644625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a40a25-1abc-44a0-89b9-1574fb7c274e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:08.649743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a40a25-1abc-44a0-89b9-1574fb7c274e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:08.664646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:08.702076 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1176/2380] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:08.706506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6d0d709b-ab1a-4c4b-92c6-7df9ef1e4680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25bc4e795b294b9f919da0548572901d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:08.707837 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a40a25-1abc-44a0-89b9-1574fb7c274e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:08.708530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6d0d709b-ab1a-4c4b-92c6-7df9ef1e4680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25bc4e795b294b9f919da0548572901d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:08.714054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e8a40a25-1abc-44a0-89b9-1574fb7c274e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:08.741471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6d0d709b-ab1a-4c4b-92c6-7df9ef1e4680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25bc4e795b294b9f919da0548572901d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:08.743418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a40a25-1abc-44a0-89b9-1574fb7c274e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:08.750662 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf15367-b88a-499e-a73c-09afd3e06c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:08.751551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf15367-b88a-499e-a73c-09afd3e06c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:08.752114 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf15367-b88a-499e-a73c-09afd3e06c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:08.752392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf15367-b88a-499e-a73c-09afd3e06c84 admin admin] PATH_INFO: `/v3/projects/21c1e895cb4b45d2b067a9dbf2ad8eda/users/63ca49f267934ae09a3ca48170115538/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:08.765759 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6d0d709b-ab1a-4c4b-92c6-7df9ef1e4680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25bc4e795b294b9f919da0548572901d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:08.767341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6d0d709b-ab1a-4c4b-92c6-7df9ef1e4680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25bc4e795b294b9f919da0548572901d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:08.784936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf15367-b88a-499e-a73c-09afd3e06c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c1e895cb4b45d2b067a9dbf2ad8eda, user_id=63ca49f267934ae09a3ca48170115538, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:08.787934 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf15367-b88a-499e-a73c-09afd3e06c84 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:08.796966 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1205/2381] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:08 2022] DELETE /identity/v3/users/25bc4e795b294b9f919da0548572901d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:14:08.801468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e627b-d0c4-4753-8546-48bc8f000fb9 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:08.801641 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e627b-d0c4-4753-8546-48bc8f000fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:08.801851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e627b-d0c4-4753-8546-48bc8f000fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:08.812456 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1177/2382] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:14:08 2022] PUT /identity/v3/projects/21c1e895cb4b45d2b067a9dbf2ad8eda/users/63ca49f267934ae09a3ca48170115538/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:08.824260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991e2e2-9242-4403-b66c-a8b03b2207dc None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:08.824554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991e2e2-9242-4403-b66c-a8b03b2207dc None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:08.825255 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991e2e2-9242-4403-b66c-a8b03b2207dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:08.827239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ab8e627b-d0c4-4753-8546-48bc8f000fb9 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:08.829273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ab8e627b-d0c4-4753-8546-48bc8f000fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:08.831571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:08.899320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9991e2e2-9242-4403-b66c-a8b03b2207dc None None] MFA Rules not processed for user `63ca49f267934ae09a3ca48170115538`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:08.900718 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9991e2e2-9242-4403-b66c-a8b03b2207dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:08.956290 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1206/2383] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:14:08.970551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e627b-d0c4-4753-8546-48bc8f000fb9 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:08.972335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ab8e627b-d0c4-4753-8546-48bc8f000fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:09.030131 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1178/2384] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:09.035358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991e2e2-9242-4403-b66c-a8b03b2207dc None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:09.052997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e627b-d0c4-4753-8546-48bc8f000fb9 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:09.057306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e405a-3a31-40e2-8acb-2aed3a469f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:09.059837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e405a-3a31-40e2-8acb-2aed3a469f24 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:09.060235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e405a-3a31-40e2-8acb-2aed3a469f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:09.060576 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e405a-3a31-40e2-8acb-2aed3a469f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:09.065405 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991e2e2-9242-4403-b66c-a8b03b2207dc None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:09.072310 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d780cd6-c801-4088-8f71-f3ae5084f02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:09.073342 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d780cd6-c801-4088-8f71-f3ae5084f02e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:09.073701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d780cd6-c801-4088-8f71-f3ae5084f02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:09.074046 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d780cd6-c801-4088-8f71-f3ae5084f02e admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:09.074986 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d780cd6-c801-4088-8f71-f3ae5084f02e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:09.077367 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d780cd6-c801-4088-8f71-f3ae5084f02e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:09.084406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e405a-3a31-40e2-8acb-2aed3a469f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:09.086080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e405a-3a31-40e2-8acb-2aed3a469f24 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:09.099171 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1207/2385] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:09.125638 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1179/2386] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:14:09 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:14:09.130346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e405a-3a31-40e2-8acb-2aed3a469f24 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:09.147035 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e405a-3a31-40e2-8acb-2aed3a469f24 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:09.231022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:09.257811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc44c96-63fb-4534-887a-fb1ebd209f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:09.263151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44c96-63fb-4534-887a-fb1ebd209f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:09.263151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44c96-63fb-4534-887a-fb1ebd209f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:09.263151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44c96-63fb-4534-887a-fb1ebd209f77 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:09.265000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc44c96-63fb-4534-887a-fb1ebd209f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:09.265948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc44c96-63fb-4534-887a-fb1ebd209f77 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:09.316795 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1208/2387] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:14:09 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:09.319368 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d780cd6-c801-4088-8f71-f3ae5084f02e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:09.335968 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d780cd6-c801-4088-8f71-f3ae5084f02e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:09.341753 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faff00a-f9bc-4e39-8f4f-288cb6075d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:09.342669 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faff00a-f9bc-4e39-8f4f-288cb6075d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:09.342957 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faff00a-f9bc-4e39-8f4f-288cb6075d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:09.343235 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faff00a-f9bc-4e39-8f4f-288cb6075d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:09.344077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faff00a-f9bc-4e39-8f4f-288cb6075d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:09.346146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faff00a-f9bc-4e39-8f4f-288cb6075d16 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:09.364151 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1180/2388] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:09.366477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc44c96-63fb-4534-887a-fb1ebd209f77 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:09.386285 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc44c96-63fb-4534-887a-fb1ebd209f77 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:09.393959 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d67d8d-3ac8-450a-bbed-60b5fe899e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:09.394393 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d67d8d-3ac8-450a-bbed-60b5fe899e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:09.394684 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d67d8d-3ac8-450a-bbed-60b5fe899e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:09.394961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d67d8d-3ac8-450a-bbed-60b5fe899e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:09.395705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d67d8d-3ac8-450a-bbed-60b5fe899e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:09.398344 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d67d8d-3ac8-450a-bbed-60b5fe899e12 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:09.416084 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1209/2389] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:14:09.417290 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faff00a-f9bc-4e39-8f4f-288cb6075d16 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:09.429682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faff00a-f9bc-4e39-8f4f-288cb6075d16 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:09.438726 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b32584-94b7-4ac4-bec9-500137b8179f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:09.440475 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b32584-94b7-4ac4-bec9-500137b8179f admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:09.440982 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b32584-94b7-4ac4-bec9-500137b8179f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:09.441461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b32584-94b7-4ac4-bec9-500137b8179f admin admin] PATH_INFO: `/v3/projects/98983a875271422894dfb9158419b8da/users/3f54c36607fd4964a1c9d0c792a33a4d/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:09.464544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b32584-94b7-4ac4-bec9-500137b8179f admin admin] RBAC: Authorizing `identity:create_grant(project_id=98983a875271422894dfb9158419b8da, user_id=3f54c36607fd4964a1c9d0c792a33a4d, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:09.468939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b32584-94b7-4ac4-bec9-500137b8179f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:09.512195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700ddbd-0000-4ef1-9ab1-b8e57c5886b9 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:09.512450 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1181/2390] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:14:09 2022] PUT /identity/v3/projects/98983a875271422894dfb9158419b8da/users/3f54c36607fd4964a1c9d0c792a33a4d/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:09.512593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700ddbd-0000-4ef1-9ab1-b8e57c5886b9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:09.512758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700ddbd-0000-4ef1-9ab1-b8e57c5886b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:09.542244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e700ddbd-0000-4ef1-9ab1-b8e57c5886b9 None None] MFA Rules not processed for user `3f54c36607fd4964a1c9d0c792a33a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:09.543524 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e700ddbd-0000-4ef1-9ab1-b8e57c5886b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:09.675823 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1210/2391] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:14:09.677238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b32584-94b7-4ac4-bec9-500137b8179f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:09.787678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b32584-94b7-4ac4-bec9-500137b8179f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:09.793972 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422e25e-41b5-495d-a704-1a12fba793f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:09.794691 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422e25e-41b5-495d-a704-1a12fba793f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:09.795332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422e25e-41b5-495d-a704-1a12fba793f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:09.795725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422e25e-41b5-495d-a704-1a12fba793f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:09.796497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422e25e-41b5-495d-a704-1a12fba793f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:09.814645 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422e25e-41b5-495d-a704-1a12fba793f5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:09.828213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700ddbd-0000-4ef1-9ab1-b8e57c5886b9 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:09.830897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e700ddbd-0000-4ef1-9ab1-b8e57c5886b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:09.857096 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700ddbd-0000-4ef1-9ab1-b8e57c5886b9 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:09.869888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:09.870628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:09.870919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:09.871199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] PATH_INFO: `/v3/projects/f81195056e23489c97e50d14fd3daf4b` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:09.872227 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1182/2392] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:14:09 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:09.873032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f81195056e23489c97e50d14fd3daf4b)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:09.873170 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422e25e-41b5-495d-a704-1a12fba793f5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:09.875026 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:09.921005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:09.989105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81195056e23489c97e50d14fd3daf4b', 'request_id': 'req-208ae8e5-bc77-49d5-81f1-1206b1ca073e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:10.004534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81195056e23489c97e50d14fd3daf4b', 'request_id': 'req-208ae8e5-bc77-49d5-81f1-1206b1ca073e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:10.020134 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81195056e23489c97e50d14fd3daf4b', 'request_id': 'req-208ae8e5-bc77-49d5-81f1-1206b1ca073e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:10.025882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422e25e-41b5-495d-a704-1a12fba793f5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:10.043347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] The token cache is being invalidate because project f81195056e23489c97e50d14fd3daf4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:10.043895 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:10.044929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:10.050987 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1211/2393] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:09 2022] DELETE /identity/v3/projects/f81195056e23489c97e50d14fd3daf4b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:14:10.052446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:10.057833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:10.167523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:10.177735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3182794-c67d-479d-bae7-f15dd07807dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:10.178554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3182794-c67d-479d-bae7-f15dd07807dd admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:10.178867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3182794-c67d-479d-bae7-f15dd07807dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:10.179179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3182794-c67d-479d-bae7-f15dd07807dd admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:10.179994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3182794-c67d-479d-bae7-f15dd07807dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:10.186035 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3182794-c67d-479d-bae7-f15dd07807dd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:10.235497 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1183/2394] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:14:09 2022] POST /identity/v3/users => generated 299 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:10.239797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3182794-c67d-479d-bae7-f15dd07807dd admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:10.245126 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c3182794-c67d-479d-bae7-f15dd07807dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:10.259966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ae8e5-bc77-49d5-81f1-1206b1ca073e admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:10.272265 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:10.272500 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3182794-c67d-479d-bae7-f15dd07807dd admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:10.273337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:10.273785 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:10.274067 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] PATH_INFO: `/v3/users/694b84e96b524fdc802e6fd0e7f1d822` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:10.275655 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=694b84e96b524fdc802e6fd0e7f1d822)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:10.279529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5059f-dcae-4285-9594-a761574bfc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:10.280480 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5059f-dcae-4285-9594-a761574bfc3b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:10.280893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5059f-dcae-4285-9594-a761574bfc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:10.281180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5059f-dcae-4285-9594-a761574bfc3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:10.281903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:10.282534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5059f-dcae-4285-9594-a761574bfc3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:10.286550 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5059f-dcae-4285-9594-a761574bfc3b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:10.308402 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1184/2395] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:10.309251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5059f-dcae-4285-9594-a761574bfc3b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:10.321717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5059f-dcae-4285-9594-a761574bfc3b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:10.326817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdf804-eebf-4fce-b226-b50cfb6a97f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:10.328481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdf804-eebf-4fce-b226-b50cfb6a97f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:10.329104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdf804-eebf-4fce-b226-b50cfb6a97f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:10.329781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdf804-eebf-4fce-b226-b50cfb6a97f3 admin admin] PATH_INFO: `/v3/projects/9b5b683c99bb4f019ecc9d15e6567d66/users/6b5b82a32e124f4c849948e9a61200f3/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:10.357221 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fdf804-eebf-4fce-b226-b50cfb6a97f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5b683c99bb4f019ecc9d15e6567d66, user_id=6b5b82a32e124f4c849948e9a61200f3, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:10.360704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fdf804-eebf-4fce-b226-b50cfb6a97f3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:10.381504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '694b84e96b524fdc802e6fd0e7f1d822'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:10.383684 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '694b84e96b524fdc802e6fd0e7f1d822'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:10.393895 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1185/2396] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:14:10 2022] PUT /identity/v3/projects/9b5b683c99bb4f019ecc9d15e6567d66/users/6b5b82a32e124f4c849948e9a61200f3/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:10.398843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdf804-eebf-4fce-b226-b50cfb6a97f3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:10.403620 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b7fdf804-eebf-4fce-b226-b50cfb6a97f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:10.422607 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '694b84e96b524fdc802e6fd0e7f1d822'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:10.436884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '694b84e96b524fdc802e6fd0e7f1d822'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:10.438504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '694b84e96b524fdc802e6fd0e7f1d822'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:10.468574 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1212/2397] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:10 2022] DELETE /identity/v3/users/694b84e96b524fdc802e6fd0e7f1d822 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:10.508874 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdf804-eebf-4fce-b226-b50cfb6a97f3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:10.545461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:10.586293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca15204-0255-40c3-a4fd-30d0c8d209e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:10.587161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca15204-0255-40c3-a4fd-30d0c8d209e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:10.587530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca15204-0255-40c3-a4fd-30d0c8d209e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:10.587901 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca15204-0255-40c3-a4fd-30d0c8d209e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:10.588743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca15204-0255-40c3-a4fd-30d0c8d209e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:10.590807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca15204-0255-40c3-a4fd-30d0c8d209e9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:10.606426 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1186/2398] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:10.607847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:10.631299 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1eee46-83ef-4460-b25b-729a44f68afe admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:10.640940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d059d-53d0-4cab-bbce-edb924c303ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:10.644944 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d059d-53d0-4cab-bbce-edb924c303ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:10.645215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d059d-53d0-4cab-bbce-edb924c303ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:10.645516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d059d-53d0-4cab-bbce-edb924c303ac admin admin] PATH_INFO: `/v3/projects/9b5b683c99bb4f019ecc9d15e6567d66/users/6b5b82a32e124f4c849948e9a61200f3/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:10.651942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d059d-53d0-4cab-bbce-edb924c303ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5b683c99bb4f019ecc9d15e6567d66, user_id=6b5b82a32e124f4c849948e9a61200f3, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:10.657132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d059d-53d0-4cab-bbce-edb924c303ac admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:10.686674 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1213/2399] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:14:10 2022] PUT /identity/v3/projects/9b5b683c99bb4f019ecc9d15e6567d66/users/6b5b82a32e124f4c849948e9a61200f3/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:14:10.698321 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b313cf-c10e-4097-8cc3-3fae09311dba None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:10.698816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b313cf-c10e-4097-8cc3-3fae09311dba None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:10.699544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b313cf-c10e-4097-8cc3-3fae09311dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:10.817752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-63b313cf-c10e-4097-8cc3-3fae09311dba None None] MFA Rules not processed for user `6b5b82a32e124f4c849948e9a61200f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:10.823060 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-63b313cf-c10e-4097-8cc3-3fae09311dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:10.981777 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1187/2400] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:10.982765 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d059d-53d0-4cab-bbce-edb924c303ac admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:10.985057 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-692d059d-53d0-4cab-bbce-edb924c303ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:11.074446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d059d-53d0-4cab-bbce-edb924c303ac admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:11.089903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89dbc4d-265a-453f-8caf-1fa3cf330aba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:11.090843 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dbc4d-265a-453f-8caf-1fa3cf330aba service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:11.091683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dbc4d-265a-453f-8caf-1fa3cf330aba service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:11.091683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dbc4d-265a-453f-8caf-1fa3cf330aba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:11.096238 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d89dbc4d-265a-453f-8caf-1fa3cf330aba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:11.185861 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89dbc4d-265a-453f-8caf-1fa3cf330aba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:11.187446 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89dbc4d-265a-453f-8caf-1fa3cf330aba service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:11.193599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:11.201000 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1214/2401] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:14:11.534795 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b313cf-c10e-4097-8cc3-3fae09311dba None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:11.536955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-63b313cf-c10e-4097-8cc3-3fae09311dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:11.750918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b313cf-c10e-4097-8cc3-3fae09311dba None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:11.760832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:11.761743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:11.762021 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:11.762322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] PATH_INFO: `/v3/projects/c3d5d605c316418caec3aa8b42374559` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:11.770156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d5d605c316418caec3aa8b42374559)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:11.772195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:11.864619 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d5d605c316418caec3aa8b42374559', 'request_id': 'req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:11.867464 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:11.877779 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d5d605c316418caec3aa8b42374559', 'request_id': 'req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:11.886462 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d5d605c316418caec3aa8b42374559', 'request_id': 'req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:11.921079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] The token cache is being invalidate because project c3d5d605c316418caec3aa8b42374559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:11.921375 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:11.922058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:11.925826 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1188/2402] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:11 2022] DELETE /identity/v3/projects/c3d5d605c316418caec3aa8b42374559 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:11.929640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89dbc4d-265a-453f-8caf-1fa3cf330aba service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:11.945310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89dbc4d-265a-453f-8caf-1fa3cf330aba service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:12.065064 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:12.067403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:12.068101 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:12.068533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] PATH_INFO: `/v3/users/a5db0f0667a2412db411a0ede5ff1a1c` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:12.070931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5db0f0667a2412db411a0ede5ff1a1c)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:12.073249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:12.171327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5db0f0667a2412db411a0ede5ff1a1c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:12.172175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5db0f0667a2412db411a0ede5ff1a1c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:12.203465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5db0f0667a2412db411a0ede5ff1a1c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:12.216529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5db0f0667a2412db411a0ede5ff1a1c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:12.217531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5db0f0667a2412db411a0ede5ff1a1c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:12.245032 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1215/2403] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:11 2022] DELETE /identity/v3/users/a5db0f0667a2412db411a0ede5ff1a1c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:14:12.978417 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:12.986485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:13.001607 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd7ff1-c287-4ca3-bc29-7f0938ee228d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:13.073657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:13.075682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:13.094151 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:13.095476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:13.096161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:13.096469 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] PATH_INFO: `/v3/projects/1a2cd0982b6a435cac7b0568c77cf92a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:13.098347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a2cd0982b6a435cac7b0568c77cf92a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:13.113413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:13.205751 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79b6b8-ac6d-444e-b94b-676dc066a221 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:13.210549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a2cd0982b6a435cac7b0568c77cf92a', 'request_id': 'req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:13.249748 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a2cd0982b6a435cac7b0568c77cf92a', 'request_id': 'req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:13.267789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a2cd0982b6a435cac7b0568c77cf92a', 'request_id': 'req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:13.285816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:13.304994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] The token cache is being invalidate because project 1a2cd0982b6a435cac7b0568c77cf92a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:13.305344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:13.307408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-86c48cba-d24e-4e5c-808b-6a2cbb8c68df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:13.316282 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1189/2404] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:12 2022] DELETE /identity/v3/projects/1a2cd0982b6a435cac7b0568c77cf92a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:13.320944 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9064b-a344-499c-98d6-d65e78f62296 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:13.321214 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9064b-a344-499c-98d6-d65e78f62296 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:13.321579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9064b-a344-499c-98d6-d65e78f62296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:13.331261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a5480-edc9-478c-90c8-5826a87dc450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:13.335434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a5480-edc9-478c-90c8-5826a87dc450 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:13.335828 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a5480-edc9-478c-90c8-5826a87dc450 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:13.336204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a5480-edc9-478c-90c8-5826a87dc450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:13.339138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d69a5480-edc9-478c-90c8-5826a87dc450 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:13.350115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b4c9064b-a344-499c-98d6-d65e78f62296 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:13.353214 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b4c9064b-a344-499c-98d6-d65e78f62296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:13.429268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a5480-edc9-478c-90c8-5826a87dc450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:13.431768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a5480-edc9-478c-90c8-5826a87dc450 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:13.448029 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1216/2405] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:14:13.494586 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1190/2406] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:14:13.502737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74875b4-08b7-4ad0-8ffd-01efdd13b9fa None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:13.503262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74875b4-08b7-4ad0-8ffd-01efdd13b9fa None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:13.503687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74875b4-08b7-4ad0-8ffd-01efdd13b9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:13.529675 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dde7ea-2df9-48a6-a366-13b5e15d3501 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:13.529970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dde7ea-2df9-48a6-a366-13b5e15d3501 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:13.530278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dde7ea-2df9-48a6-a366-13b5e15d3501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:13.536346 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b74875b4-08b7-4ad0-8ffd-01efdd13b9fa None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:13.539505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b74875b4-08b7-4ad0-8ffd-01efdd13b9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:13.553364 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1217/2407] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:13.555208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74875b4-08b7-4ad0-8ffd-01efdd13b9fa None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:13.565128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74875b4-08b7-4ad0-8ffd-01efdd13b9fa None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:13.571081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef72188-a769-4d4b-89c8-11120242267f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:13.573296 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:13.574456 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef72188-a769-4d4b-89c8-11120242267f admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:13.575020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef72188-a769-4d4b-89c8-11120242267f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:13.575499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef72188-a769-4d4b-89c8-11120242267f admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:13.577181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef72188-a769-4d4b-89c8-11120242267f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:14:13.579956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef72188-a769-4d4b-89c8-11120242267f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:13.582695 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef72188-a769-4d4b-89c8-11120242267f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:13.600267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-22dde7ea-2df9-48a6-a366-13b5e15d3501 None None] MFA Rules not processed for user `721521f50380490ea84fb46734f0f592`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:13.602074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-22dde7ea-2df9-48a6-a366-13b5e15d3501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:13.609061 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1218/2408] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:14:13.610814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef72188-a769-4d4b-89c8-11120242267f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:13.625212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef72188-a769-4d4b-89c8-11120242267f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:13.633012 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0148c-e229-4b19-add6-418e67d4def3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:13.634199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0148c-e229-4b19-add6-418e67d4def3 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:13.635065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0148c-e229-4b19-add6-418e67d4def3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:13.635708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0148c-e229-4b19-add6-418e67d4def3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:13.637220 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0148c-e229-4b19-add6-418e67d4def3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:13.642156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0148c-e229-4b19-add6-418e67d4def3 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:13.718524 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1219/2409] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:14:13 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:13.727241 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0148c-e229-4b19-add6-418e67d4def3 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:13.882751 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1191/2410] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:13.900136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dde7ea-2df9-48a6-a366-13b5e15d3501 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:13.901657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-22dde7ea-2df9-48a6-a366-13b5e15d3501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:13.936348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0148c-e229-4b19-add6-418e67d4def3 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:13.955964 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fe84b-cdbd-4eea-9781-b210302f8e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:13.957755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fe84b-cdbd-4eea-9781-b210302f8e97 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:13.958822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fe84b-cdbd-4eea-9781-b210302f8e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:13.960932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fe84b-cdbd-4eea-9781-b210302f8e97 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:13.962365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fe84b-cdbd-4eea-9781-b210302f8e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:13.966193 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fe84b-cdbd-4eea-9781-b210302f8e97 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:14.004251 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1220/2411] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:14:13 2022] POST /identity/v3/users => generated 286 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:14.008795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fe84b-cdbd-4eea-9781-b210302f8e97 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:14.035602 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fe84b-cdbd-4eea-9781-b210302f8e97 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:14.051153 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4e689-e9d5-4fb4-a992-d6604669eb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:14.051864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4e689-e9d5-4fb4-a992-d6604669eb98 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:14.052148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4e689-e9d5-4fb4-a992-d6604669eb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:14.052424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4e689-e9d5-4fb4-a992-d6604669eb98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:14.053177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4e689-e9d5-4fb4-a992-d6604669eb98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:14.059397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4e689-e9d5-4fb4-a992-d6604669eb98 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:14.073142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dde7ea-2df9-48a6-a366-13b5e15d3501 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:14.090618 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1221/2412] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:14.091715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4e689-e9d5-4fb4-a992-d6604669eb98 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:14.108735 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4e689-e9d5-4fb4-a992-d6604669eb98 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:14.114957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d266f-b88f-4cd4-9a97-16576faf02e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:14.115691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d266f-b88f-4cd4-9a97-16576faf02e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:14.115981 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d266f-b88f-4cd4-9a97-16576faf02e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:14.116263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d266f-b88f-4cd4-9a97-16576faf02e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:14.117098 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d266f-b88f-4cd4-9a97-16576faf02e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:14.118833 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d266f-b88f-4cd4-9a97-16576faf02e7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:14.131264 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1222/2413] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:14.135393 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d266f-b88f-4cd4-9a97-16576faf02e7 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:14.152630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d266f-b88f-4cd4-9a97-16576faf02e7 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:14.158830 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff5411-9018-4326-8261-b3bcf1993ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:14.159901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff5411-9018-4326-8261-b3bcf1993ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:14.160247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff5411-9018-4326-8261-b3bcf1993ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:14.160608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff5411-9018-4326-8261-b3bcf1993ac8 admin admin] PATH_INFO: `/v3/projects/561156ae2b574f9f89e54c68160aca4d/users/0312d4f1dad04fae971555d477d1c528/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:14.174382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:14.183285 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff5411-9018-4326-8261-b3bcf1993ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=561156ae2b574f9f89e54c68160aca4d, user_id=0312d4f1dad04fae971555d477d1c528, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:14.187396 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff5411-9018-4326-8261-b3bcf1993ac8 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:14.212397 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1223/2414] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:14:14 2022] PUT /identity/v3/projects/561156ae2b574f9f89e54c68160aca4d/users/0312d4f1dad04fae971555d477d1c528/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:14:14.220099 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa755d14-a704-4043-adb2-f11eec0ee2b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:14.220757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa755d14-a704-4043-adb2-f11eec0ee2b6 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:14.222382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa755d14-a704-4043-adb2-f11eec0ee2b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:14.222922 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa755d14-a704-4043-adb2-f11eec0ee2b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:14.224090 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999fa89-afa8-46be-8617-b581337ed298 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:14.224428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999fa89-afa8-46be-8617-b581337ed298 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:14.224703 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999fa89-afa8-46be-8617-b581337ed298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:14.227081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-aa755d14-a704-4043-adb2-f11eec0ee2b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:14.269748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a999fa89-afa8-46be-8617-b581337ed298 None None] MFA Rules not processed for user `0312d4f1dad04fae971555d477d1c528`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:14.273506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a999fa89-afa8-46be-8617-b581337ed298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:14.275980 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:14.323026 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa755d14-a704-4043-adb2-f11eec0ee2b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:14.325458 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa755d14-a704-4043-adb2-f11eec0ee2b6 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:14.339477 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1192/2415] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:13 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:14.399426 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1224/2416] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:14.411201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa755d14-a704-4043-adb2-f11eec0ee2b6 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:14.412684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-aa755d14-a704-4043-adb2-f11eec0ee2b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:14.488309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa755d14-a704-4043-adb2-f11eec0ee2b6 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:14.544639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d062fb-0796-4868-887d-3c91136989b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:14.545296 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d062fb-0796-4868-887d-3c91136989b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:14.545635 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d062fb-0796-4868-887d-3c91136989b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:14.545851 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d062fb-0796-4868-887d-3c91136989b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:14.555886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d062fb-0796-4868-887d-3c91136989b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:14.559746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d062fb-0796-4868-887d-3c91136989b2 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:14.567431 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999fa89-afa8-46be-8617-b581337ed298 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:14.569075 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a999fa89-afa8-46be-8617-b581337ed298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:14.571266 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1193/2417] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:14.583922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999fa89-afa8-46be-8617-b581337ed298 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:14.588362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33e1d3-0215-4f96-80f4-cc9865c54ff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:14.589119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33e1d3-0215-4f96-80f4-cc9865c54ff1 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:14.589343 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33e1d3-0215-4f96-80f4-cc9865c54ff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:14.589617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33e1d3-0215-4f96-80f4-cc9865c54ff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:14.591580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8c33e1d3-0215-4f96-80f4-cc9865c54ff1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:14.609788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d062fb-0796-4868-887d-3c91136989b2 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:14.672903 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33e1d3-0215-4f96-80f4-cc9865c54ff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:14.674454 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33e1d3-0215-4f96-80f4-cc9865c54ff1 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:14.684249 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1225/2418] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:14:14.692987 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d062fb-0796-4868-887d-3c91136989b2 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:14.700470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:14.701454 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:14.701896 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:14.702179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] PATH_INFO: `/v3/users/0312d4f1dad04fae971555d477d1c528` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:14.703805 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] RBAC: Authorizing `identity:delete_user(user_id=0312d4f1dad04fae971555d477d1c528)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:14.706082 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:14.778157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:14.794332 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0312d4f1dad04fae971555d477d1c528'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:14.795523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0312d4f1dad04fae971555d477d1c528'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:14.796411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0312d4f1dad04fae971555d477d1c528'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:14.805054 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0312d4f1dad04fae971555d477d1c528'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:14.820984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0312d4f1dad04fae971555d477d1c528'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:14.837587 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1194/2419] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:14 2022] DELETE /identity/v3/users/0312d4f1dad04fae971555d477d1c528 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:14.842271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69700-e191-4cbc-ace9-dfcbf59ce01f None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:14.842612 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69700-e191-4cbc-ace9-dfcbf59ce01f None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:14.842890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69700-e191-4cbc-ace9-dfcbf59ce01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:14.862841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-15d69700-e191-4cbc-ace9-dfcbf59ce01f None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:14.864195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-15d69700-e191-4cbc-ace9-dfcbf59ce01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:14.943303 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1226/2420] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:14:14.951179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:14.952646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:15.016233 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe649b4-6fe5-4c48-903e-17e20edb2fff admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:15.022087 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3e5b0-cd76-4a4d-8481-5d1b31fdd886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:15.024349 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3e5b0-cd76-4a4d-8481-5d1b31fdd886 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:15.024349 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3e5b0-cd76-4a4d-8481-5d1b31fdd886 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:15.024349 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3e5b0-cd76-4a4d-8481-5d1b31fdd886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:15.035064 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3e5b0-cd76-4a4d-8481-5d1b31fdd886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:15.036705 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3e5b0-cd76-4a4d-8481-5d1b31fdd886 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:15.047477 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1195/2421] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:15.127860 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d69700-e191-4cbc-ace9-dfcbf59ce01f None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:15.130261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-15d69700-e191-4cbc-ace9-dfcbf59ce01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:15.160521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d69700-e191-4cbc-ace9-dfcbf59ce01f None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:15.177051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:15.177941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:15.178246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:15.178530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] PATH_INFO: `/v3/users/dd6cc8cb22bb4adca166abfb731ebd04` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:15.179976 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd6cc8cb22bb4adca166abfb731ebd04)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:15.181957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:15.199420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:15.278682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd6cc8cb22bb4adca166abfb731ebd04'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:15.279362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd6cc8cb22bb4adca166abfb731ebd04'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:15.294381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd6cc8cb22bb4adca166abfb731ebd04'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:15.316390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd6cc8cb22bb4adca166abfb731ebd04'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:15.327846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd6cc8cb22bb4adca166abfb731ebd04'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:15.355747 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1227/2422] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:15 2022] DELETE /identity/v3/users/dd6cc8cb22bb4adca166abfb731ebd04 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:15.359785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a00cf-ab45-4b7b-af3f-c034a684a2e3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:15.360875 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a00cf-ab45-4b7b-af3f-c034a684a2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:15.361305 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a00cf-ab45-4b7b-af3f-c034a684a2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:15.384385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-893a00cf-ab45-4b7b-af3f-c034a684a2e3 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:15.386146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-893a00cf-ab45-4b7b-af3f-c034a684a2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:15.462178 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1196/2423] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:14:15.480181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:15.481706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:15.483161 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a00cf-ab45-4b7b-af3f-c034a684a2e3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:15.485509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-893a00cf-ab45-4b7b-af3f-c034a684a2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:15.509665 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083c2d6-bcfb-4a64-81c0-b4fcd078df16 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:15.520572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:15.521497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:15.521783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:15.522061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] PATH_INFO: `/v3/projects/561156ae2b574f9f89e54c68160aca4d` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:15.523835 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=561156ae2b574f9f89e54c68160aca4d)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:15.525955 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:15.674993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a00cf-ab45-4b7b-af3f-c034a684a2e3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:15.681216 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:15.708761 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '561156ae2b574f9f89e54c68160aca4d', 'request_id': 'req-993e6117-4f6b-4de8-aa47-cc28cb1d2138'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:15.716947 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '561156ae2b574f9f89e54c68160aca4d', 'request_id': 'req-993e6117-4f6b-4de8-aa47-cc28cb1d2138'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:15.731698 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '561156ae2b574f9f89e54c68160aca4d', 'request_id': 'req-993e6117-4f6b-4de8-aa47-cc28cb1d2138'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:15.761993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] The token cache is being invalidate because project 561156ae2b574f9f89e54c68160aca4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:15.762372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:15.764157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-993e6117-4f6b-4de8-aa47-cc28cb1d2138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:15.769004 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1228/2424] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:15 2022] DELETE /identity/v3/projects/561156ae2b574f9f89e54c68160aca4d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:15.776490 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34b1f4-8971-429b-b2e0-bab807330a04 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:15.776790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34b1f4-8971-429b-b2e0-bab807330a04 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:15.777056 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34b1f4-8971-429b-b2e0-bab807330a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:15.777333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c42786-0e14-47a8-95f4-824538e943a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:15.778028 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c42786-0e14-47a8-95f4-824538e943a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:15.778372 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c42786-0e14-47a8-95f4-824538e943a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:15.778756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c42786-0e14-47a8-95f4-824538e943a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:15.780658 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-66c42786-0e14-47a8-95f4-824538e943a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:15.800507 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-7f34b1f4-8971-429b-b2e0-bab807330a04 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:15.801877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7f34b1f4-8971-429b-b2e0-bab807330a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:15.880415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c42786-0e14-47a8-95f4-824538e943a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:15.881749 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1229/2425] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:15.882314 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c42786-0e14-47a8-95f4-824538e943a2 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:15.891111 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dad979-9c16-45ee-9653-b8644d027289 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:15.891474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dad979-9c16-45ee-9653-b8644d027289 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:15.891821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dad979-9c16-45ee-9653-b8644d027289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:15.894841 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1197/2426] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:15.896204 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d8053-c504-47d0-ad43-20d3b38bcf44 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:15.896644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d8053-c504-47d0-ad43-20d3b38bcf44 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:15.896979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d8053-c504-47d0-ad43-20d3b38bcf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:15.912765 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-89dad979-9c16-45ee-9653-b8644d027289 None None] MFA Rules not processed for user `9966a96281b745e684d0cd16bb470e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:15.914349 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-89dad979-9c16-45ee-9653-b8644d027289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:15.919558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fd1d8053-c504-47d0-ad43-20d3b38bcf44 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:15.920586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:15.922083 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fd1d8053-c504-47d0-ad43-20d3b38bcf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:15.932563 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1198/2427] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:15.937125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d8053-c504-47d0-ad43-20d3b38bcf44 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:15.952958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d8053-c504-47d0-ad43-20d3b38bcf44 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:15.960024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251e140-4cf5-4257-a88f-6d2bf89728a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:15.960935 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e140-4cf5-4257-a88f-6d2bf89728a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:15.961216 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e140-4cf5-4257-a88f-6d2bf89728a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:15.961588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e140-4cf5-4257-a88f-6d2bf89728a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:15.962478 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251e140-4cf5-4257-a88f-6d2bf89728a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:14:15.962864 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251e140-4cf5-4257-a88f-6d2bf89728a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:15.964688 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251e140-4cf5-4257-a88f-6d2bf89728a0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:15.993620 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1199/2428] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 10:14:16.006854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251e140-4cf5-4257-a88f-6d2bf89728a0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:16.015989 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1230/2429] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:16.018869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251e140-4cf5-4257-a88f-6d2bf89728a0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:16.025303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50531350-9ef7-4dfa-8a2a-d7592817b720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:16.025989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50531350-9ef7-4dfa-8a2a-d7592817b720 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:16.026345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50531350-9ef7-4dfa-8a2a-d7592817b720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:16.026625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50531350-9ef7-4dfa-8a2a-d7592817b720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:16.027410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50531350-9ef7-4dfa-8a2a-d7592817b720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:16.029539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50531350-9ef7-4dfa-8a2a-d7592817b720 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:16.064939 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1200/2430] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:14:16 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:16.069665 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dad979-9c16-45ee-9653-b8644d027289 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:16.083593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dad979-9c16-45ee-9653-b8644d027289 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:16.232654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51c3df-e576-4d83-8c1e-e18b80192b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:16.233348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51c3df-e576-4d83-8c1e-e18b80192b9f admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:16.233783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51c3df-e576-4d83-8c1e-e18b80192b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:16.234133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51c3df-e576-4d83-8c1e-e18b80192b9f admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:16.234927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51c3df-e576-4d83-8c1e-e18b80192b9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:16.237216 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51c3df-e576-4d83-8c1e-e18b80192b9f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:16.300253 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1231/2431] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:14:16 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:14:16.304429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50531350-9ef7-4dfa-8a2a-d7592817b720 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:16.318573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50531350-9ef7-4dfa-8a2a-d7592817b720 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:16.332440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a605b-e5c2-4d52-92da-ca99824c6e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:16.333257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a605b-e5c2-4d52-92da-ca99824c6e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:16.333781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a605b-e5c2-4d52-92da-ca99824c6e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:16.334184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a605b-e5c2-4d52-92da-ca99824c6e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:16.335309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a605b-e5c2-4d52-92da-ca99824c6e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:16.339352 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a605b-e5c2-4d52-92da-ca99824c6e6f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:16.356105 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1201/2432] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:16.360428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51c3df-e576-4d83-8c1e-e18b80192b9f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:16.372693 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51c3df-e576-4d83-8c1e-e18b80192b9f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:16.380533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77127d87-1874-4d4f-8ce3-3213f55efafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:16.381372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77127d87-1874-4d4f-8ce3-3213f55efafc admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:16.381720 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77127d87-1874-4d4f-8ce3-3213f55efafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:16.386954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77127d87-1874-4d4f-8ce3-3213f55efafc admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:16.387860 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77127d87-1874-4d4f-8ce3-3213f55efafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:16.389594 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77127d87-1874-4d4f-8ce3-3213f55efafc admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:16.411171 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1232/2433] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:16.415125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a605b-e5c2-4d52-92da-ca99824c6e6f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:16.431076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a605b-e5c2-4d52-92da-ca99824c6e6f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:16.439690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed2594-f808-486b-b926-70d1f213ff03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:16.441394 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed2594-f808-486b-b926-70d1f213ff03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:16.441841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed2594-f808-486b-b926-70d1f213ff03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:16.442537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed2594-f808-486b-b926-70d1f213ff03 admin admin] PATH_INFO: `/v3/projects/24e072409112479b8d428dbb6d7a7963/users/95fb9318372747e8a77264ae58b19c47/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:16.445993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77127d87-1874-4d4f-8ce3-3213f55efafc admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:16.449252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-77127d87-1874-4d4f-8ce3-3213f55efafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:16.467783 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed2594-f808-486b-b926-70d1f213ff03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e072409112479b8d428dbb6d7a7963, user_id=95fb9318372747e8a77264ae58b19c47, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:16.471256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed2594-f808-486b-b926-70d1f213ff03 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:16.483870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77127d87-1874-4d4f-8ce3-3213f55efafc admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:16.494845 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1202/2434] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:14:16 2022] PUT /identity/v3/projects/24e072409112479b8d428dbb6d7a7963/users/95fb9318372747e8a77264ae58b19c47/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:16.505771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427ad74-0af7-4b22-9a31-825b196b291e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:16.506262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427ad74-0af7-4b22-9a31-825b196b291e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:16.506551 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427ad74-0af7-4b22-9a31-825b196b291e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:16.551928 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c427ad74-0af7-4b22-9a31-825b196b291e None None] MFA Rules not processed for user `95fb9318372747e8a77264ae58b19c47`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:16.553205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c427ad74-0af7-4b22-9a31-825b196b291e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:16.594036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:16.599600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:16.600503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:16.600858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:16.601137 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] PATH_INFO: `/v3/projects/9e0cf774c7a4462b90c943c89bc07c97` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:16.602784 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0cf774c7a4462b90c943c89bc07c97)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:16.605646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:16.673847 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1203/2435] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:16.683057 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:16.685598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427ad74-0af7-4b22-9a31-825b196b291e None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:16.688777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c427ad74-0af7-4b22-9a31-825b196b291e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:16.713570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0cf774c7a4462b90c943c89bc07c97', 'request_id': 'req-56034bbb-0861-4bd5-adf4-742dcd72c19c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:16.725208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0cf774c7a4462b90c943c89bc07c97', 'request_id': 'req-56034bbb-0861-4bd5-adf4-742dcd72c19c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:16.739669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0cf774c7a4462b90c943c89bc07c97', 'request_id': 'req-56034bbb-0861-4bd5-adf4-742dcd72c19c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:16.767423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] The token cache is being invalidate because project 9e0cf774c7a4462b90c943c89bc07c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:16.767687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:16.769095 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:16.771202 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1233/2436] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:16 2022] DELETE /identity/v3/projects/9e0cf774c7a4462b90c943c89bc07c97 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:14:16.812451 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427ad74-0af7-4b22-9a31-825b196b291e None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:16.896530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe449887-434f-4202-b1e0-147316e30430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:16.897218 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449887-434f-4202-b1e0-147316e30430 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:16.897649 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449887-434f-4202-b1e0-147316e30430 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:16.898238 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449887-434f-4202-b1e0-147316e30430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:16.900121 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fe449887-434f-4202-b1e0-147316e30430 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:16.974256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe449887-434f-4202-b1e0-147316e30430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:16.975993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe449887-434f-4202-b1e0-147316e30430 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:16.988755 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1204/2437] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:14:18.987869 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:18.988341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:19.114815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56034bbb-0861-4bd5-adf4-742dcd72c19c admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:19.123270 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:19.124322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:19.124673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:19.125015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] PATH_INFO: `/v3/users/63ca49f267934ae09a3ca48170115538` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:19.126710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ca49f267934ae09a3ca48170115538)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:19.134979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:19.204920 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ca49f267934ae09a3ca48170115538'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.205603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ca49f267934ae09a3ca48170115538'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.219196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ca49f267934ae09a3ca48170115538'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.228298 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ca49f267934ae09a3ca48170115538'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.229039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ca49f267934ae09a3ca48170115538'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.247354 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1234/2438] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:18 2022] DELETE /identity/v3/users/63ca49f267934ae09a3ca48170115538 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:19.252034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b06efd-7bf0-4622-800b-bafda796bd91 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:19.252590 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b06efd-7bf0-4622-800b-bafda796bd91 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:19.252686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b06efd-7bf0-4622-800b-bafda796bd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:19.270947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-b8b06efd-7bf0-4622-800b-bafda796bd91 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:19.272176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b8b06efd-7bf0-4622-800b-bafda796bd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:19.313477 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:19.335091 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1205/2439] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:19.344913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:19.346423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:19.394623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:19.408626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c2943-a27d-453e-9b9b-48ee6c365fbe admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:19.412810 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c791b-777e-4a86-b01f-ec3f885bdd25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:19.413496 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c791b-777e-4a86-b01f-ec3f885bdd25 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:19.413782 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c791b-777e-4a86-b01f-ec3f885bdd25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:19.414065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c791b-777e-4a86-b01f-ec3f885bdd25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:19.424188 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c791b-777e-4a86-b01f-ec3f885bdd25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:19.425678 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c791b-777e-4a86-b01f-ec3f885bdd25 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:19.436057 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1235/2440] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:14:19.740426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b06efd-7bf0-4622-800b-bafda796bd91 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:19.741911 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-b8b06efd-7bf0-4622-800b-bafda796bd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:19.762915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b06efd-7bf0-4622-800b-bafda796bd91 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:19.767907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:19.768924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:19.769114 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:19.769392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] PATH_INFO: `/v3/projects/21c1e895cb4b45d2b067a9dbf2ad8eda` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:19.770734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c1e895cb4b45d2b067a9dbf2ad8eda)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:19.772797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:19.819638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c1e895cb4b45d2b067a9dbf2ad8eda', 'request_id': 'req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.827777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c1e895cb4b45d2b067a9dbf2ad8eda', 'request_id': 'req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.833653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c1e895cb4b45d2b067a9dbf2ad8eda', 'request_id': 'req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.844731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] The token cache is being invalidate because project 21c1e895cb4b45d2b067a9dbf2ad8eda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:19.845018 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.845648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.847250 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1206/2441] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:19 2022] DELETE /identity/v3/projects/21c1e895cb4b45d2b067a9dbf2ad8eda => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:19.851421 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c791b-777e-4a86-b01f-ec3f885bdd25 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:19.861564 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c791b-777e-4a86-b01f-ec3f885bdd25 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:19.921046 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:19.921934 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:19.922245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:19.922629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] PATH_INFO: `/v3/users/3f54c36607fd4964a1c9d0c792a33a4d` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:19.923829 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f54c36607fd4964a1c9d0c792a33a4d)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:19.925785 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:19.980719 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f54c36607fd4964a1c9d0c792a33a4d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.981447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f54c36607fd4964a1c9d0c792a33a4d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:19.994161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f54c36607fd4964a1c9d0c792a33a4d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.001442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f54c36607fd4964a1c9d0c792a33a4d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.002198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f54c36607fd4964a1c9d0c792a33a4d'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.016860 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1236/2442] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:19 2022] DELETE /identity/v3/users/3f54c36607fd4964a1c9d0c792a33a4d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:20.300573 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:20.302565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:20.330056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06831b-b4c2-4307-8bfe-3aabfe3ecf38 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:20.418963 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:20.419872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:20.420163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:20.422120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] PATH_INFO: `/v3/projects/98983a875271422894dfb9158419b8da` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:20.427100 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98983a875271422894dfb9158419b8da)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:20.429511 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:20.494754 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98983a875271422894dfb9158419b8da', 'request_id': 'req-df2aae48-62a0-4ff7-9289-af8d7daca681'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.502263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98983a875271422894dfb9158419b8da', 'request_id': 'req-df2aae48-62a0-4ff7-9289-af8d7daca681'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.508690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98983a875271422894dfb9158419b8da', 'request_id': 'req-df2aae48-62a0-4ff7-9289-af8d7daca681'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.511003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:20.521857 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] The token cache is being invalidate because project 98983a875271422894dfb9158419b8da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:20.522428 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.523391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.524961 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1207/2443] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:20 2022] DELETE /identity/v3/projects/98983a875271422894dfb9158419b8da => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:20.528950 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:20.530422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:20.543667 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee949f84-224f-4c85-97e8-919361ab2337 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:20.606704 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0f814-aeef-472c-a8f1-5d1a9606b436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:20.608010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0f814-aeef-472c-a8f1-5d1a9606b436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:20.608504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0f814-aeef-472c-a8f1-5d1a9606b436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:20.608885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0f814-aeef-472c-a8f1-5d1a9606b436 admin admin] PATH_INFO: `/v3/users/6b5b82a32e124f4c849948e9a61200f3` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:20.610163 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0f814-aeef-472c-a8f1-5d1a9606b436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b5b82a32e124f4c849948e9a61200f3)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:20.612614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0f814-aeef-472c-a8f1-5d1a9606b436 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:20.619130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:20.668966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5ee0f814-aeef-472c-a8f1-5d1a9606b436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b5b82a32e124f4c849948e9a61200f3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.669659 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5ee0f814-aeef-472c-a8f1-5d1a9606b436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b5b82a32e124f4c849948e9a61200f3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.689914 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5ee0f814-aeef-472c-a8f1-5d1a9606b436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b5b82a32e124f4c849948e9a61200f3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.697159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5ee0f814-aeef-472c-a8f1-5d1a9606b436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b5b82a32e124f4c849948e9a61200f3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.697883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5ee0f814-aeef-472c-a8f1-5d1a9606b436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b5b82a32e124f4c849948e9a61200f3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:20.713637 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1237/2444] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:20 2022] DELETE /identity/v3/users/6b5b82a32e124f4c849948e9a61200f3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:21.058409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:21.060072 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:21.074327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aae48-62a0-4ff7-9289-af8d7daca681 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:21.142371 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:21.143256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:21.143543 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:21.143818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] PATH_INFO: `/v3/projects/9b5b683c99bb4f019ecc9d15e6567d66` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:21.145222 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b5b683c99bb4f019ecc9d15e6567d66)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:21.147268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:21.208242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b5b683c99bb4f019ecc9d15e6567d66', 'request_id': 'req-1671bada-93a4-4d82-8106-d0e9e84e7794'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:21.217065 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b5b683c99bb4f019ecc9d15e6567d66', 'request_id': 'req-1671bada-93a4-4d82-8106-d0e9e84e7794'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:21.223880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b5b683c99bb4f019ecc9d15e6567d66', 'request_id': 'req-1671bada-93a4-4d82-8106-d0e9e84e7794'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:21.237246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] The token cache is being invalidate because project 9b5b683c99bb4f019ecc9d15e6567d66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:21.237554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:21.238946 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-1671bada-93a4-4d82-8106-d0e9e84e7794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:21.240953 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1208/2445] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:21 2022] DELETE /identity/v3/projects/9b5b683c99bb4f019ecc9d15e6567d66 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:21.245907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d19f1d-06da-4768-ba8a-96f9e6743c06 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:21.246932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d19f1d-06da-4768-ba8a-96f9e6743c06 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:21.247519 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d19f1d-06da-4768-ba8a-96f9e6743c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:21.271420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-34d19f1d-06da-4768-ba8a-96f9e6743c06 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:21.273154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-34d19f1d-06da-4768-ba8a-96f9e6743c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:21.338809 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1238/2446] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:21.346708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a4148-ab24-4719-ac23-96848c037b60 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:21.347195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a4148-ab24-4719-ac23-96848c037b60 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:21.347317 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a4148-ab24-4719-ac23-96848c037b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:21.366864 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2d1a4148-ab24-4719-ac23-96848c037b60 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:21.368078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2d1a4148-ab24-4719-ac23-96848c037b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:21.376594 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1209/2447] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:14:21.380956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d19f1d-06da-4768-ba8a-96f9e6743c06 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:21.391422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d19f1d-06da-4768-ba8a-96f9e6743c06 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:21.396417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbf743-13c4-481d-8d44-bb1cc79566a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:21.397219 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbf743-13c4-481d-8d44-bb1cc79566a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:21.397574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbf743-13c4-481d-8d44-bb1cc79566a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:21.397930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbf743-13c4-481d-8d44-bb1cc79566a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:21.398771 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbf743-13c4-481d-8d44-bb1cc79566a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:14:21.399107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbf743-13c4-481d-8d44-bb1cc79566a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:21.400936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbf743-13c4-481d-8d44-bb1cc79566a7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:21.413295 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1239/2448] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:14:21.417715 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a4148-ab24-4719-ac23-96848c037b60 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:21.427350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a4148-ab24-4719-ac23-96848c037b60 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:21.432791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76572bda-2e07-4c0a-8408-3770f047459e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:21.433453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76572bda-2e07-4c0a-8408-3770f047459e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:21.433732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76572bda-2e07-4c0a-8408-3770f047459e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:21.434005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76572bda-2e07-4c0a-8408-3770f047459e admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:21.434738 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76572bda-2e07-4c0a-8408-3770f047459e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:21.436644 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76572bda-2e07-4c0a-8408-3770f047459e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:21.465268 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1210/2449] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:14:21 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:21.471384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbf743-13c4-481d-8d44-bb1cc79566a7 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:21.482609 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbf743-13c4-481d-8d44-bb1cc79566a7 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:21.541046 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d6914-1e58-4158-80b1-c1756e60b946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:21.541747 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d6914-1e58-4158-80b1-c1756e60b946 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:21.542026 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d6914-1e58-4158-80b1-c1756e60b946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:21.542326 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d6914-1e58-4158-80b1-c1756e60b946 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:21.543017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d6914-1e58-4158-80b1-c1756e60b946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:21.544943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d6914-1e58-4158-80b1-c1756e60b946 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:21.560972 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:21.570316 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1240/2450] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:14:21 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:14:21.571539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76572bda-2e07-4c0a-8408-3770f047459e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:21.582938 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76572bda-2e07-4c0a-8408-3770f047459e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:21.588576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4535699-783b-48fc-8a50-511d65dfe8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:21.589272 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4535699-783b-48fc-8a50-511d65dfe8fb admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:21.589552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4535699-783b-48fc-8a50-511d65dfe8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:21.589869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4535699-783b-48fc-8a50-511d65dfe8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:21.590584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4535699-783b-48fc-8a50-511d65dfe8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:21.592174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4535699-783b-48fc-8a50-511d65dfe8fb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:21.605956 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1211/2451] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:21.607225 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d6914-1e58-4158-80b1-c1756e60b946 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:21.618107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d6914-1e58-4158-80b1-c1756e60b946 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:21.623490 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda10409-126a-497b-b96d-ca26bf8c7537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:21.624167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda10409-126a-497b-b96d-ca26bf8c7537 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:21.624449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda10409-126a-497b-b96d-ca26bf8c7537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:21.624724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda10409-126a-497b-b96d-ca26bf8c7537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:21.625453 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda10409-126a-497b-b96d-ca26bf8c7537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:21.627112 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda10409-126a-497b-b96d-ca26bf8c7537 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:21.640667 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1241/2452] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:14:21.641909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4535699-783b-48fc-8a50-511d65dfe8fb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:21.653789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4535699-783b-48fc-8a50-511d65dfe8fb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:21.659431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7ab3e-6907-497a-8c38-b4c39b3cda5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:21.660410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7ab3e-6907-497a-8c38-b4c39b3cda5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:21.660767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7ab3e-6907-497a-8c38-b4c39b3cda5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:21.661112 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7ab3e-6907-497a-8c38-b4c39b3cda5a admin admin] PATH_INFO: `/v3/projects/6add3e1ca68e4ab3aedb4f1cdf4698f8/users/135b98edc240424ea17a325160ac106c/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:21.686109 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7ab3e-6907-497a-8c38-b4c39b3cda5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6add3e1ca68e4ab3aedb4f1cdf4698f8, user_id=135b98edc240424ea17a325160ac106c, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:21.687655 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7ab3e-6907-497a-8c38-b4c39b3cda5a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:21.702179 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1212/2453] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:14:21 2022] PUT /identity/v3/projects/6add3e1ca68e4ab3aedb4f1cdf4698f8/users/135b98edc240424ea17a325160ac106c/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:21.713037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f343b-1614-434d-9370-548aae12f651 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:21.713351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f343b-1614-434d-9370-548aae12f651 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:21.713631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f343b-1614-434d-9370-548aae12f651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:21.740714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-2b7f343b-1614-434d-9370-548aae12f651 None None] MFA Rules not processed for user `135b98edc240424ea17a325160ac106c`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:21.741978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2b7f343b-1614-434d-9370-548aae12f651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:21.824022 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1242/2454] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:21.836159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7ab3e-6907-497a-8c38-b4c39b3cda5a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:21.838071 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-85d7ab3e-6907-497a-8c38-b4c39b3cda5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:21.864518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:21.900914 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7ab3e-6907-497a-8c38-b4c39b3cda5a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:21.907717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e85d0-e5fb-43f3-848f-e1acc832936e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b97736e49e2a4bfb9cf7093cfa8b2b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:21.908785 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e85d0-e5fb-43f3-848f-e1acc832936e service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:21.909268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e85d0-e5fb-43f3-848f-e1acc832936e service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:21.909744 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e85d0-e5fb-43f3-848f-e1acc832936e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:21.919901 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e85d0-e5fb-43f3-848f-e1acc832936e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:21.921387 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e85d0-e5fb-43f3-848f-e1acc832936e service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:21.930927 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1213/2455] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:23.943344 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e190c-0eb4-48f5-a9d4-ce26902f6d94 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:23.943822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e190c-0eb4-48f5-a9d4-ce26902f6d94 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:23.944080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e190c-0eb4-48f5-a9d4-ce26902f6d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:23.962470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3d6e190c-0eb4-48f5-a9d4-ce26902f6d94 None None] MFA Rules not processed for user `3c6e5faddabd4208b6e476be4b59e834`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:23.963852 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3d6e190c-0eb4-48f5-a9d4-ce26902f6d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:24.034116 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1243/2456] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:27.767299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e85d0-e5fb-43f3-848f-e1acc832936e service glance] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:27.768990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-486e85d0-e5fb-43f3-848f-e1acc832936e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:27.838089 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e85d0-e5fb-43f3-848f-e1acc832936e service glance] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:27.842963 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:27.843877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:27.844165 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:27.844447 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] PATH_INFO: `/v3/users/95fb9318372747e8a77264ae58b19c47` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:27.845650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95fb9318372747e8a77264ae58b19c47)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:27.847645 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:27.895713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95fb9318372747e8a77264ae58b19c47'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:27.896427 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95fb9318372747e8a77264ae58b19c47'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:27.897066 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95fb9318372747e8a77264ae58b19c47'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:27.904318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95fb9318372747e8a77264ae58b19c47'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:27.917449 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95fb9318372747e8a77264ae58b19c47'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:27.936442 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1214/2457] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:27 2022] DELETE /identity/v3/users/95fb9318372747e8a77264ae58b19c47 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:27.941653 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddb73e-a93b-4011-8444-25d092441b56 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:27.942124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddb73e-a93b-4011-8444-25d092441b56 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:27.942278 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddb73e-a93b-4011-8444-25d092441b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:27.963188 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-bfddb73e-a93b-4011-8444-25d092441b56 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:27.964356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-bfddb73e-a93b-4011-8444-25d092441b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:28.024006 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1244/2458] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:28.034250 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:28.035781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:28.085176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:28.100676 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30514e3f-4b0c-4630-a3ac-fac3271e4925 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:28.105275 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e480aee-f418-4aee-8d66-52aa990bc5c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:28.105976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480aee-f418-4aee-8d66-52aa990bc5c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:28.106299 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480aee-f418-4aee-8d66-52aa990bc5c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:28.106578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480aee-f418-4aee-8d66-52aa990bc5c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:28.116938 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e480aee-f418-4aee-8d66-52aa990bc5c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:28.118459 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e480aee-f418-4aee-8d66-52aa990bc5c2 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:28.129018 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1215/2459] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:14:28.416562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddb73e-a93b-4011-8444-25d092441b56 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:28.418283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-bfddb73e-a93b-4011-8444-25d092441b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:28.430790 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddb73e-a93b-4011-8444-25d092441b56 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:28.436409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:28.437326 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:28.437607 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:28.437878 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] PATH_INFO: `/v3/projects/24e072409112479b8d428dbb6d7a7963` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:28.439264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24e072409112479b8d428dbb6d7a7963)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:28.441225 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:28.463566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:28.495913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e072409112479b8d428dbb6d7a7963', 'request_id': 'req-1be04e02-45ba-4fdf-8e53-61f8a6c82549'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:28.503854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e072409112479b8d428dbb6d7a7963', 'request_id': 'req-1be04e02-45ba-4fdf-8e53-61f8a6c82549'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:28.510542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e072409112479b8d428dbb6d7a7963', 'request_id': 'req-1be04e02-45ba-4fdf-8e53-61f8a6c82549'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:28.525845 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] The token cache is being invalidate because project 24e072409112479b8d428dbb6d7a7963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:14:28.526163 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:28.526806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1be04e02-45ba-4fdf-8e53-61f8a6c82549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:28.528370 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1245/2460] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:14:28 2022] DELETE /identity/v3/projects/24e072409112479b8d428dbb6d7a7963 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:38.124202 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:38.523023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:59.467786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e480aee-f418-4aee-8d66-52aa990bc5c2 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:59.470062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7e480aee-f418-4aee-8d66-52aa990bc5c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:59.555032 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e480aee-f418-4aee-8d66-52aa990bc5c2 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:59.612448 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:59.613338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:59.613642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:59.613994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] PATH_INFO: `/v3/users/135b98edc240424ea17a325160ac106c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:59.615674 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=135b98edc240424ea17a325160ac106c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:59.617966 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:59.666436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '135b98edc240424ea17a325160ac106c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:59.667326 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '135b98edc240424ea17a325160ac106c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:59.667939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '135b98edc240424ea17a325160ac106c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:59.675159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '135b98edc240424ea17a325160ac106c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:59.687302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '135b98edc240424ea17a325160ac106c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:14:59.702586 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1216/2461] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:14:59 2022] DELETE /identity/v3/users/135b98edc240424ea17a325160ac106c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:14:59.707164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19245e14-a040-4b5f-920b-fb765967d0f7 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:59.707656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19245e14-a040-4b5f-920b-fb765967d0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:59.707751 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19245e14-a040-4b5f-920b-fb765967d0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:59.736989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-19245e14-a040-4b5f-920b-fb765967d0f7 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:14:59.739028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-19245e14-a040-4b5f-920b-fb765967d0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:59.807580 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1246/2462] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:14:59.815777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:14:59.817959 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:14:59.852228 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:59.933445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64131d-9134-42a3-a174-9798525e15f8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:14:59.937587 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635fa23-3250-4460-ac00-4a507f6acaff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:14:59.938261 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635fa23-3250-4460-ac00-4a507f6acaff service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:14:59.938540 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635fa23-3250-4460-ac00-4a507f6acaff service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:14:59.938853 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635fa23-3250-4460-ac00-4a507f6acaff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:14:59.949025 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635fa23-3250-4460-ac00-4a507f6acaff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:14:59.950558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635fa23-3250-4460-ac00-4a507f6acaff service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:14:59.960896 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1217/2463] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:14:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:15:00.200088 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19245e14-a040-4b5f-920b-fb765967d0f7 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:15:00.210739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19245e14-a040-4b5f-920b-fb765967d0f7 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:15:00.214844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471a78b-5a23-4ca1-b856-9be14b119056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:15:00.215442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a78b-5a23-4ca1-b856-9be14b119056 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:00.215746 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a78b-5a23-4ca1-b856-9be14b119056 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:00.216024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a78b-5a23-4ca1-b856-9be14b119056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:00.217912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8471a78b-5a23-4ca1-b856-9be14b119056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:00.267394 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635fa23-3250-4460-ac00-4a507f6acaff service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:15:00.269319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3635fa23-3250-4460-ac00-4a507f6acaff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:00.286012 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635fa23-3250-4460-ac00-4a507f6acaff service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:15:00.292599 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:15:00.293743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:00.294122 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:00.294782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] PATH_INFO: `/v3/projects/6add3e1ca68e4ab3aedb4f1cdf4698f8` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:00.296257 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6add3e1ca68e4ab3aedb4f1cdf4698f8)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:15:00.296768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471a78b-5a23-4ca1-b856-9be14b119056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:15:00.298180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471a78b-5a23-4ca1-b856-9be14b119056 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:15:00.299099 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:15:00.308472 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1247/2464] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:15:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:15:00.353506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6add3e1ca68e4ab3aedb4f1cdf4698f8', 'request_id': 'req-6615c404-d8e4-49f6-9de3-0508b5238266'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:15:00.362686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6add3e1ca68e4ab3aedb4f1cdf4698f8', 'request_id': 'req-6615c404-d8e4-49f6-9de3-0508b5238266'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:15:00.368525 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6add3e1ca68e4ab3aedb4f1cdf4698f8', 'request_id': 'req-6615c404-d8e4-49f6-9de3-0508b5238266'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:15:00.380192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] The token cache is being invalidate because project 6add3e1ca68e4ab3aedb4f1cdf4698f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:15:00.380623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:15:00.381328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:15:00.383051 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1218/2465] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:15:00 2022] DELETE /identity/v3/projects/6add3e1ca68e4ab3aedb4f1cdf4698f8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:15:10.304069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=60 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:10.377174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=61,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:34.314645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061476f-b25f-400a-ad65-7a3097b56e07 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:34.315194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061476f-b25f-400a-ad65-7a3097b56e07 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:34.315194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061476f-b25f-400a-ad65-7a3097b56e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:34.377941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-4061476f-b25f-400a-ad65-7a3097b56e07 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:15:34.379201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4061476f-b25f-400a-ad65-7a3097b56e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:34.453838 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1248/2466] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:15:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:15:34.512753 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:15:34.514350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:34.612568 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615c404-d8e4-49f6-9de3-0508b5238266 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:15:34.618553 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3b9e1-bd06-4eda-886c-5a43d98a0cbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:15:34.619239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3b9e1-bd06-4eda-886c-5a43d98a0cbd service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:34.619494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3b9e1-bd06-4eda-886c-5a43d98a0cbd service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:34.619802 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3b9e1-bd06-4eda-886c-5a43d98a0cbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:34.633097 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3b9e1-bd06-4eda-886c-5a43d98a0cbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:15:34.635722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3b9e1-bd06-4eda-886c-5a43d98a0cbd service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:15:34.644199 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1219/2467] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:15:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:15:44.450474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=47 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:44.640682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=41 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:48.428215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061476f-b25f-400a-ad65-7a3097b56e07 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:15:48.430858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4061476f-b25f-400a-ad65-7a3097b56e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:48.541592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061476f-b25f-400a-ad65-7a3097b56e07 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:15:48.547220 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f95c-24d2-4e04-9754-9f1f82c798ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:15:48.547872 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f95c-24d2-4e04-9754-9f1f82c798ac service placement] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:48.548149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f95c-24d2-4e04-9754-9f1f82c798ac service placement] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:48.548430 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f95c-24d2-4e04-9754-9f1f82c798ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:48.550557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c6f0f95c-24d2-4e04-9754-9f1f82c798ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:48.562395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f95c-24d2-4e04-9754-9f1f82c798ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:15:48.563993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f95c-24d2-4e04-9754-9f1f82c798ac service placement] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:15:48.573896 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1249/2468] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:15:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:15:50.696677 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bf324-1e71-497b-a0dc-ddc6d42607ac None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:50.697157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bf324-1e71-497b-a0dc-ddc6d42607ac None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:50.697510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bf324-1e71-497b-a0dc-ddc6d42607ac None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:50.699379 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1220/2469] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:15:50.705284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea903c-708a-4bd1-9bae-a78a312ba58b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:50.705777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea903c-708a-4bd1-9bae-a78a312ba58b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:50.705876 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea903c-708a-4bd1-9bae-a78a312ba58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:50.770391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-87ea903c-708a-4bd1-9bae-a78a312ba58b None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:15:50.771613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-87ea903c-708a-4bd1-9bae-a78a312ba58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:50.849334 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1250/2470] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:15:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:15:50.861362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bf324-1e71-497b-a0dc-ddc6d42607ac None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:15:50.863273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-474bf324-1e71-497b-a0dc-ddc6d42607ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:50.950527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bf324-1e71-497b-a0dc-ddc6d42607ac None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:15:50.956650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f9b90-5235-4856-99a8-dfdf61973fd4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b97736e49e2a4bfb9cf7093cfa8b2b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:15:50.957313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9b90-5235-4856-99a8-dfdf61973fd4 service glance] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:50.957595 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9b90-5235-4856-99a8-dfdf61973fd4 service glance] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:50.957869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9b90-5235-4856-99a8-dfdf61973fd4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:50.968755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f9b90-5235-4856-99a8-dfdf61973fd4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:15:50.970191 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f9b90-5235-4856-99a8-dfdf61973fd4 service glance] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:15:50.980578 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1221/2471] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:15:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:15:51.055335 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf23b6-46b2-40e8-8a4c-f5c7154606c4 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:51.055655 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf23b6-46b2-40e8-8a4c-f5c7154606c4 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:51.055901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf23b6-46b2-40e8-8a4c-f5c7154606c4 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:51.057285 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1251/2472] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:15:51.062617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fb63c-6e37-4ea7-a05a-f32b40626bd3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:51.062943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fb63c-6e37-4ea7-a05a-f32b40626bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:51.063219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fb63c-6e37-4ea7-a05a-f32b40626bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:51.081789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7c8fb63c-6e37-4ea7-a05a-f32b40626bd3 None None] MFA Rules not processed for user `e0865f21da0a4c95a84c0099a1b9a78b`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:15:51.083034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7c8fb63c-6e37-4ea7-a05a-f32b40626bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:51.133779 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1222/2473] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:15:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:15:51.145955 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf23b6-46b2-40e8-8a4c-f5c7154606c4 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:15:51.147816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c0bf23b6-46b2-40e8-8a4c-f5c7154606c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:51.161546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf23b6-46b2-40e8-8a4c-f5c7154606c4 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:15:51.165356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88050574-9ad4-49b0-9587-ae9ee43fbd25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:15:51.165993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88050574-9ad4-49b0-9587-ae9ee43fbd25 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:51.166369 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88050574-9ad4-49b0-9587-ae9ee43fbd25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:51.166646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88050574-9ad4-49b0-9587-ae9ee43fbd25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:51.174203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88050574-9ad4-49b0-9587-ae9ee43fbd25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:15:51.175681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88050574-9ad4-49b0-9587-ae9ee43fbd25 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:15:51.183433 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1252/2474] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:15:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:15:51.265286 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401c6c7-dcd9-48d1-96ab-976adcdc44a3 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:51.265579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401c6c7-dcd9-48d1-96ab-976adcdc44a3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:51.265858 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401c6c7-dcd9-48d1-96ab-976adcdc44a3 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:51.267339 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1223/2475] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:15:51.272797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad909a51-fbe8-4b34-a8e1-375ece775292 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:51.273077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad909a51-fbe8-4b34-a8e1-375ece775292 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:51.273380 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad909a51-fbe8-4b34-a8e1-375ece775292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:51.286766 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:51.292996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ad909a51-fbe8-4b34-a8e1-375ece775292 None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:15:51.294209 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ad909a51-fbe8-4b34-a8e1-375ece775292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:51.305007 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1253/2476] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:15:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:15:51.315815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401c6c7-dcd9-48d1-96ab-976adcdc44a3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:15:51.327964 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401c6c7-dcd9-48d1-96ab-976adcdc44a3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:15:51.331916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ff064-7a03-4ea0-8e00-2b993a77fa61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:15:51.332544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ff064-7a03-4ea0-8e00-2b993a77fa61 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:51.332826 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ff064-7a03-4ea0-8e00-2b993a77fa61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:51.333101 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ff064-7a03-4ea0-8e00-2b993a77fa61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:51.343953 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ff064-7a03-4ea0-8e00-2b993a77fa61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:15:51.345379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ff064-7a03-4ea0-8e00-2b993a77fa61 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:15:51.356142 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1224/2477] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:15:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:15:53.653637 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbb2a0-767a-4213-b7e0-f97991106722 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:53.654125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbb2a0-767a-4213-b7e0-f97991106722 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:53.654235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbb2a0-767a-4213-b7e0-f97991106722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:53.672227 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3dfbb2a0-767a-4213-b7e0-f97991106722 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:15:53.673408 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3dfbb2a0-767a-4213-b7e0-f97991106722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:15:53.681089 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1254/2478] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:15:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:15:53.693005 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ff064-7a03-4ea0-8e00-2b993a77fa61 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:15:53.705150 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ff064-7a03-4ea0-8e00-2b993a77fa61 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:15:53.711499 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb9fef-e01a-41f5-b333-34f1071336ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:15:53.712177 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb9fef-e01a-41f5-b333-34f1071336ee service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:15:53.712481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb9fef-e01a-41f5-b333-34f1071336ee service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:15:53.712752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb9fef-e01a-41f5-b333-34f1071336ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:15:53.721724 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb9fef-e01a-41f5-b333-34f1071336ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:15:53.723419 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb9fef-e01a-41f5-b333-34f1071336ee service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:15:53.731749 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1225/2479] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:15:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:16:03.667190 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:03.728426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=70 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:15.052253 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267a6aa-dd6d-4f2e-811b-532eba517e02 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:16:15.052758 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267a6aa-dd6d-4f2e-811b-532eba517e02 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:16:15.052875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267a6aa-dd6d-4f2e-811b-532eba517e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:16:15.071600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-6267a6aa-dd6d-4f2e-811b-532eba517e02 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:16:15.072880 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6267a6aa-dd6d-4f2e-811b-532eba517e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:16:15.080565 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1255/2480] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jul 5 10:16:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:16:15.094249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb9fef-e01a-41f5-b333-34f1071336ee service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:16:15.105282 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb9fef-e01a-41f5-b333-34f1071336ee service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:16:15.110740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a113c82-a133-4ec8-87ce-eb9820de7633 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:16:15.111526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a113c82-a133-4ec8-87ce-eb9820de7633 service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:16:15.111899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a113c82-a133-4ec8-87ce-eb9820de7633 service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:16:15.112180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a113c82-a133-4ec8-87ce-eb9820de7633 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:16:15.120731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a113c82-a133-4ec8-87ce-eb9820de7633 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:16:15.122352 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a113c82-a133-4ec8-87ce-eb9820de7633 service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:16:15.130926 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1226/2481] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:16:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:16:15.316919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179105eb-c322-4fe9-a827-9fa894518831 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:16:15.317207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179105eb-c322-4fe9-a827-9fa894518831 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:16:15.317640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179105eb-c322-4fe9-a827-9fa894518831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:16:15.339887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-179105eb-c322-4fe9-a827-9fa894518831 None None] MFA Rules not processed for user `ae9cd6d6d5824aa1932d3353578285af`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:16:15.341135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-179105eb-c322-4fe9-a827-9fa894518831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:16:15.399652 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1256/2482] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:16:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:16:25.127648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:25.392757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=28 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:39.157827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a29be-a44a-402e-ac97-073ce5c82f12 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:16:39.158393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a29be-a44a-402e-ac97-073ce5c82f12 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:16:39.158393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a29be-a44a-402e-ac97-073ce5c82f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:16:39.176854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8e6a29be-a44a-402e-ac97-073ce5c82f12 None None] MFA Rules not processed for user `3c6e5faddabd4208b6e476be4b59e834`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:16:39.178091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8e6a29be-a44a-402e-ac97-073ce5c82f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:16:39.237727 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1227/2483] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:16:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:16:49.229736 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=25 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:24.602978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179105eb-c322-4fe9-a827-9fa894518831 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:17:24.604797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-179105eb-c322-4fe9-a827-9fa894518831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:17:24.619659 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179105eb-c322-4fe9-a827-9fa894518831 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:17:24.624952 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e9b6d-4320-476a-bd1c-a371936cf5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:17:24.625873 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e9b6d-4320-476a-bd1c-a371936cf5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:17:24.626143 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e9b6d-4320-476a-bd1c-a371936cf5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:17:24.626445 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e9b6d-4320-476a-bd1c-a371936cf5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:17:24.628449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-964e9b6d-4320-476a-bd1c-a371936cf5b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:17:24.642898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e9b6d-4320-476a-bd1c-a371936cf5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:17:24.644336 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e9b6d-4320-476a-bd1c-a371936cf5b8 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:17:24.655453 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1257/2484] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:17:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:17:25.857462 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a29be-a44a-402e-ac97-073ce5c82f12 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:17:25.868356 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a29be-a44a-402e-ac97-073ce5c82f12 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:17:25.872936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d8832-5624-4e73-a2dd-5435a5a9da58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:17:25.873586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d8832-5624-4e73-a2dd-5435a5a9da58 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:17:25.873867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d8832-5624-4e73-a2dd-5435a5a9da58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:17:25.874152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d8832-5624-4e73-a2dd-5435a5a9da58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:17:25.876327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-654d8832-5624-4e73-a2dd-5435a5a9da58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:17:25.890578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d8832-5624-4e73-a2dd-5435a5a9da58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:17:25.892208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d8832-5624-4e73-a2dd-5435a5a9da58 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:17:25.903207 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1228/2485] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:17:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:17:32.363170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d97972-625b-4f48-8eb7-557273feab62 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:17:32.363707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d97972-625b-4f48-8eb7-557273feab62 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:17:32.363821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d97972-625b-4f48-8eb7-557273feab62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:17:32.386544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-62d97972-625b-4f48-8eb7-557273feab62 None None] MFA Rules not processed for user `51e20b9a489342d2b8f0f85542ab370c`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:17:32.387774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-62d97972-625b-4f48-8eb7-557273feab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:17:32.443465 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1258/2486] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:17:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:17:35.898366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:39.997113 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d8832-5624-4e73-a2dd-5435a5a9da58 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:17:39.999841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-654d8832-5624-4e73-a2dd-5435a5a9da58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:17:40.021635 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d8832-5624-4e73-a2dd-5435a5a9da58 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:17:40.028653 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac25ba9-5e31-41f5-a75f-112222576123 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:17:40.029294 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac25ba9-5e31-41f5-a75f-112222576123 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:17:40.029575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac25ba9-5e31-41f5-a75f-112222576123 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:17:40.029851 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac25ba9-5e31-41f5-a75f-112222576123 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:17:40.031961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2ac25ba9-5e31-41f5-a75f-112222576123 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:17:40.043517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac25ba9-5e31-41f5-a75f-112222576123 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:17:40.045103 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac25ba9-5e31-41f5-a75f-112222576123 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:17:40.054648 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1229/2487] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:17:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:17:42.436672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=31 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:48.136575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d97972-625b-4f48-8eb7-557273feab62 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:17:48.138205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-62d97972-625b-4f48-8eb7-557273feab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:17:48.218942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d97972-625b-4f48-8eb7-557273feab62 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:17:48.222837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd09c3-2696-43c0-8be8-73bc2e0e7c85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:17:48.223611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd09c3-2696-43c0-8be8-73bc2e0e7c85 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:17:48.223882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd09c3-2696-43c0-8be8-73bc2e0e7c85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:17:48.224156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd09c3-2696-43c0-8be8-73bc2e0e7c85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:17:48.226080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ffbd09c3-2696-43c0-8be8-73bc2e0e7c85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:17:48.237225 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd09c3-2696-43c0-8be8-73bc2e0e7c85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:17:48.238682 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd09c3-2696-43c0-8be8-73bc2e0e7c85 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:17:48.247960 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1259/2488] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:17:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:17:50.050291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:54.394663 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac25ba9-5e31-41f5-a75f-112222576123 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:17:54.406583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac25ba9-5e31-41f5-a75f-112222576123 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:17:54.413095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f16351-48e8-4ab3-b2ff-bc9975ce30b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:17:54.414288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f16351-48e8-4ab3-b2ff-bc9975ce30b2 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:17:54.414288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f16351-48e8-4ab3-b2ff-bc9975ce30b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:17:54.414522 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f16351-48e8-4ab3-b2ff-bc9975ce30b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:17:54.416578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-30f16351-48e8-4ab3-b2ff-bc9975ce30b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:17:54.430733 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f16351-48e8-4ab3-b2ff-bc9975ce30b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:17:54.433302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f16351-48e8-4ab3-b2ff-bc9975ce30b2 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:17:54.444776 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1230/2489] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:17:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:17:58.243732 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=35 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:04.440097 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:07.475549 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd09c3-2696-43c0-8be8-73bc2e0e7c85 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:07.489222 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd09c3-2696-43c0-8be8-73bc2e0e7c85 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:07.494742 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1b1cc-c726-4371-95ed-377a60027498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:07.495440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1b1cc-c726-4371-95ed-377a60027498 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:07.495729 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1b1cc-c726-4371-95ed-377a60027498 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:07.495989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1b1cc-c726-4371-95ed-377a60027498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:07.498341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2ea1b1cc-c726-4371-95ed-377a60027498 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:07.509120 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1b1cc-c726-4371-95ed-377a60027498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:07.510658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1b1cc-c726-4371-95ed-377a60027498 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:07.519327 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1260/2490] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:08.210922 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f16351-48e8-4ab3-b2ff-bc9975ce30b2 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:08.212595 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-30f16351-48e8-4ab3-b2ff-bc9975ce30b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:08.243479 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f16351-48e8-4ab3-b2ff-bc9975ce30b2 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:08.251003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9d82a-ad1b-41b9-81a8-209e2151b357 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:08.251735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9d82a-ad1b-41b9-81a8-209e2151b357 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:08.252058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9d82a-ad1b-41b9-81a8-209e2151b357 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:08.252307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9d82a-ad1b-41b9-81a8-209e2151b357 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:08.254235 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-44c9d82a-ad1b-41b9-81a8-209e2151b357 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:08.272085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9d82a-ad1b-41b9-81a8-209e2151b357 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:08.274003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9d82a-ad1b-41b9-81a8-209e2151b357 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:08.286081 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1231/2491] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:09.674639 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1b1cc-c726-4371-95ed-377a60027498 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:09.683984 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1b1cc-c726-4371-95ed-377a60027498 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:09.688458 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa956c4c-37fb-4e35-aee9-aa79fef2895a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:09.689145 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa956c4c-37fb-4e35-aee9-aa79fef2895a service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:09.689429 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa956c4c-37fb-4e35-aee9-aa79fef2895a service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:09.689707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa956c4c-37fb-4e35-aee9-aa79fef2895a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:09.691736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-fa956c4c-37fb-4e35-aee9-aa79fef2895a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:09.757923 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa956c4c-37fb-4e35-aee9-aa79fef2895a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:09.760337 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa956c4c-37fb-4e35-aee9-aa79fef2895a service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:09.770684 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1261/2492] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:11.948024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0375fbb-d1ec-4752-9c9d-cdd378a3e655 None None] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:11.948608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0375fbb-d1ec-4752-9c9d-cdd378a3e655 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:11.948608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0375fbb-d1ec-4752-9c9d-cdd378a3e655 None None] PATH_INFO: `/` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:11.949977 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1232/2493] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:18:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:18:11.958312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc19a0e-8bb2-4422-b2f5-410437f0702c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:11.958904 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc19a0e-8bb2-4422-b2f5-410437f0702c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:11.959160 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc19a0e-8bb2-4422-b2f5-410437f0702c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:11.982281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-efc19a0e-8bb2-4422-b2f5-410437f0702c None None] MFA Rules not processed for user `e0865f21da0a4c95a84c0099a1b9a78b`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:11.984009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-efc19a0e-8bb2-4422-b2f5-410437f0702c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:12.000044 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1262/2494] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:18:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:12.027048 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0375fbb-d1ec-4752-9c9d-cdd378a3e655 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:12.042497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0375fbb-d1ec-4752-9c9d-cdd378a3e655 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:12.046553 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783d353-dff8-421b-be24-cfc97f684d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:12.047533 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d353-dff8-421b-be24-cfc97f684d95 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:12.048024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d353-dff8-421b-be24-cfc97f684d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:12.048645 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d353-dff8-421b-be24-cfc97f684d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:12.060683 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783d353-dff8-421b-be24-cfc97f684d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:12.062801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783d353-dff8-421b-be24-cfc97f684d95 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:12.082989 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1233/2495] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:12.199516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bf603-3004-43cf-bfbf-1786b0432fd2 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:12.200066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bf603-3004-43cf-bfbf-1786b0432fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:12.200330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bf603-3004-43cf-bfbf-1786b0432fd2 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:12.201783 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1263/2496] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:18:12.207709 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286187b-1fb4-4151-93ff-4b6393b30f99 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:12.208003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286187b-1fb4-4151-93ff-4b6393b30f99 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:12.208410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286187b-1fb4-4151-93ff-4b6393b30f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:12.229481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8286187b-1fb4-4151-93ff-4b6393b30f99 None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:12.230729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8286187b-1fb4-4151-93ff-4b6393b30f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:12.240969 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1234/2497] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:18:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:12.252095 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bf603-3004-43cf-bfbf-1786b0432fd2 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:12.263638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bf603-3004-43cf-bfbf-1786b0432fd2 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:12.267205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04871ed0-637f-40cc-be0c-060e4b897a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:12.267946 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04871ed0-637f-40cc-be0c-060e4b897a35 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:12.268238 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04871ed0-637f-40cc-be0c-060e4b897a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:12.268681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04871ed0-637f-40cc-be0c-060e4b897a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:12.279278 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04871ed0-637f-40cc-be0c-060e4b897a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:12.280741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04871ed0-637f-40cc-be0c-060e4b897a35 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:12.291996 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1264/2498] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:18:12.354288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286187b-1fb4-4151-93ff-4b6393b30f99 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:12.369254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286187b-1fb4-4151-93ff-4b6393b30f99 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:12.375855 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460cc239-338f-4f78-a06b-c1c3f9a24359 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:12.376524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cc239-338f-4f78-a06b-c1c3f9a24359 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:12.376808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cc239-338f-4f78-a06b-c1c3f9a24359 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:12.377084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cc239-338f-4f78-a06b-c1c3f9a24359 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:12.379139 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-460cc239-338f-4f78-a06b-c1c3f9a24359 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:12.392377 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460cc239-338f-4f78-a06b-c1c3f9a24359 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:12.394719 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460cc239-338f-4f78-a06b-c1c3f9a24359 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:12.407715 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1235/2499] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:13.729362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04871ed0-637f-40cc-be0c-060e4b897a35 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:13.744882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04871ed0-637f-40cc-be0c-060e4b897a35 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:13.750752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50680ef9-3c32-4d63-9d67-946646352f10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:13.752776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50680ef9-3c32-4d63-9d67-946646352f10 service placement] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:13.753246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50680ef9-3c32-4d63-9d67-946646352f10 service placement] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:13.753681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50680ef9-3c32-4d63-9d67-946646352f10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:13.755787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-50680ef9-3c32-4d63-9d67-946646352f10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:13.773187 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50680ef9-3c32-4d63-9d67-946646352f10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:13.775156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50680ef9-3c32-4d63-9d67-946646352f10 service placement] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:13.786639 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1265/2500] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:13.813633 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460cc239-338f-4f78-a06b-c1c3f9a24359 service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:13.817419 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-460cc239-338f-4f78-a06b-c1c3f9a24359 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:13.832811 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460cc239-338f-4f78-a06b-c1c3f9a24359 service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:13.840971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:13.843092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:13.843652 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:13.843936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] PATH_INFO: `/v3/users/3c6e5faddabd4208b6e476be4b59e834` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:13.845525 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c6e5faddabd4208b6e476be4b59e834)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:13.847548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:13.915396 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c6e5faddabd4208b6e476be4b59e834'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:13.916056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c6e5faddabd4208b6e476be4b59e834'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:13.916709 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c6e5faddabd4208b6e476be4b59e834'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:13.924333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c6e5faddabd4208b6e476be4b59e834'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:13.938899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c6e5faddabd4208b6e476be4b59e834'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:13.961811 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1236/2501] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:13 2022] DELETE /identity/v3/users/3c6e5faddabd4208b6e476be4b59e834 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:14.153407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955873c7-9e88-48e3-916e-a22eb8f8baf5 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:14.153690 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955873c7-9e88-48e3-916e-a22eb8f8baf5 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:14.153983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955873c7-9e88-48e3-916e-a22eb8f8baf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:14.173207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-955873c7-9e88-48e3-916e-a22eb8f8baf5 None None] MFA Rules not processed for user `ae9cd6d6d5824aa1932d3353578285af`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:14.174528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-955873c7-9e88-48e3-916e-a22eb8f8baf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:14.233817 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1266/2502] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:14.379127 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:14.380705 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:14.474617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7c9ff-845d-4e4d-915d-0d9f153b27b9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:14.480612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:14.480996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:14.481278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:14.481701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] PATH_INFO: `/v3/projects/de80b31cd6c144ac838b613648f8f3a7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:14.487904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] RBAC: Authorizing `identity:delete_project(project_id=de80b31cd6c144ac838b613648f8f3a7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:14.490478 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:14.536882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:14.555358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de80b31cd6c144ac838b613648f8f3a7', 'request_id': 'req-94fde18d-30ac-41c4-805a-7859a767002f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:14.580251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de80b31cd6c144ac838b613648f8f3a7', 'request_id': 'req-94fde18d-30ac-41c4-805a-7859a767002f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:14.588075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de80b31cd6c144ac838b613648f8f3a7', 'request_id': 'req-94fde18d-30ac-41c4-805a-7859a767002f'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:14.596494 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955873c7-9e88-48e3-916e-a22eb8f8baf5 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:14.599993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-955873c7-9e88-48e3-916e-a22eb8f8baf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:14.602803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] The token cache is being invalidate because project de80b31cd6c144ac838b613648f8f3a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:14.603099 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:14.603843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-94fde18d-30ac-41c4-805a-7859a767002f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:14.605807 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1237/2503] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:14 2022] DELETE /identity/v3/projects/de80b31cd6c144ac838b613648f8f3a7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:14.611331 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038669-ca27-43b4-96ff-47af89b8dd06 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:14.617889 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038669-ca27-43b4-96ff-47af89b8dd06 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:14.618183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038669-ca27-43b4-96ff-47af89b8dd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:14.677945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1a038669-ca27-43b4-96ff-47af89b8dd06 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:14.679187 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1a038669-ca27-43b4-96ff-47af89b8dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:14.703460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955873c7-9e88-48e3-916e-a22eb8f8baf5 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:14.709988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:14.754360 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1238/2504] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:14.756354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a038669-ca27-43b4-96ff-47af89b8dd06 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:14.761255 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1a038669-ca27-43b4-96ff-47af89b8dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:14.771992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed52891-c2dd-4ca1-84f4-c25a07471b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:14.772817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed52891-c2dd-4ca1-84f4-c25a07471b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:14.773489 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed52891-c2dd-4ca1-84f4-c25a07471b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:14.774534 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed52891-c2dd-4ca1-84f4-c25a07471b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:14.776616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-bed52891-c2dd-4ca1-84f4-c25a07471b39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:14.790723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed52891-c2dd-4ca1-84f4-c25a07471b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:14.792708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed52891-c2dd-4ca1-84f4-c25a07471b39 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:14.802708 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1267/2505] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:14.804184 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d2f32-e68c-46e7-9873-536bf42df2bf None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:14.804647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d2f32-e68c-46e7-9873-536bf42df2bf None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:14.804931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d2f32-e68c-46e7-9873-536bf42df2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:14.822969 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a71d2f32-e68c-46e7-9873-536bf42df2bf None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:14.824414 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a71d2f32-e68c-46e7-9873-536bf42df2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:14.836567 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1268/2506] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:14.837814 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d2f32-e68c-46e7-9873-536bf42df2bf None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:14.847977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d2f32-e68c-46e7-9873-536bf42df2bf None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:14.853661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c174b-c4fb-4a24-a9af-8ba48bfd130e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:14.854368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c174b-c4fb-4a24-a9af-8ba48bfd130e admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:14.854647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c174b-c4fb-4a24-a9af-8ba48bfd130e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:14.855311 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c174b-c4fb-4a24-a9af-8ba48bfd130e admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:14.856379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c174b-c4fb-4a24-a9af-8ba48bfd130e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:18:14.856838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c174b-c4fb-4a24-a9af-8ba48bfd130e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:14.863030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a038669-ca27-43b4-96ff-47af89b8dd06 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:14.863924 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c174b-c4fb-4a24-a9af-8ba48bfd130e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:14.868958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576240f8-bccc-40b4-a178-0e8d2e6c13e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:14.869620 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576240f8-bccc-40b4-a178-0e8d2e6c13e0 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:14.869906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576240f8-bccc-40b4-a178-0e8d2e6c13e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:14.870188 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576240f8-bccc-40b4-a178-0e8d2e6c13e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:14.872962 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-576240f8-bccc-40b4-a178-0e8d2e6c13e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:14.881427 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1269/2507] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:18:14.885930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c174b-c4fb-4a24-a9af-8ba48bfd130e admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:14.892424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576240f8-bccc-40b4-a178-0e8d2e6c13e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:14.894445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576240f8-bccc-40b4-a178-0e8d2e6c13e0 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:14.897700 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c174b-c4fb-4a24-a9af-8ba48bfd130e admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:14.902569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72a81f-5980-4787-887b-9c7929ce5b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:14.902989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72a81f-5980-4787-887b-9c7929ce5b8d admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:14.903273 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72a81f-5980-4787-887b-9c7929ce5b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:14.903605 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72a81f-5980-4787-887b-9c7929ce5b8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:14.904334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72a81f-5980-4787-887b-9c7929ce5b8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:14.905938 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1239/2508] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:14.906475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72a81f-5980-4787-887b-9c7929ce5b8d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:14.936735 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1270/2509] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:14 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:18:14.941367 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576240f8-bccc-40b4-a178-0e8d2e6c13e0 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:14.952157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576240f8-bccc-40b4-a178-0e8d2e6c13e0 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:14.977410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:15.014600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cd459-7ebc-4467-81f3-2050e1960a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:15.015290 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cd459-7ebc-4467-81f3-2050e1960a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.015570 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cd459-7ebc-4467-81f3-2050e1960a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.015849 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cd459-7ebc-4467-81f3-2050e1960a27 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.016695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cd459-7ebc-4467-81f3-2050e1960a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:15.018662 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cd459-7ebc-4467-81f3-2050e1960a27 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:15.039676 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1240/2510] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:14 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:15.043888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72a81f-5980-4787-887b-9c7929ce5b8d admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:15.054521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72a81f-5980-4787-887b-9c7929ce5b8d admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:15.059266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e5580-9ecc-4ca0-804d-00ddee3d2bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:15.059952 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e5580-9ecc-4ca0-804d-00ddee3d2bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.060304 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e5580-9ecc-4ca0-804d-00ddee3d2bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.060580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e5580-9ecc-4ca0-804d-00ddee3d2bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.061305 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e5580-9ecc-4ca0-804d-00ddee3d2bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:15.062975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e5580-9ecc-4ca0-804d-00ddee3d2bb6 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:15.073692 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1271/2511] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:15.077991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cd459-7ebc-4467-81f3-2050e1960a27 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:15.089327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cd459-7ebc-4467-81f3-2050e1960a27 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:15.095547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d97d7-1ba5-41c5-9265-c45781ef5fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:15.096244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d97d7-1ba5-41c5-9265-c45781ef5fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.096534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d97d7-1ba5-41c5-9265-c45781ef5fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.096808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d97d7-1ba5-41c5-9265-c45781ef5fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.097555 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d97d7-1ba5-41c5-9265-c45781ef5fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:15.099218 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d97d7-1ba5-41c5-9265-c45781ef5fd0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:15.111010 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1241/2512] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:15.115110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e5580-9ecc-4ca0-804d-00ddee3d2bb6 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:15.125820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e5580-9ecc-4ca0-804d-00ddee3d2bb6 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:15.130771 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86edbc4-8316-49a8-a2f6-1a7e80cfedcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:15.131672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86edbc4-8316-49a8-a2f6-1a7e80cfedcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.131939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86edbc4-8316-49a8-a2f6-1a7e80cfedcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.132210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86edbc4-8316-49a8-a2f6-1a7e80cfedcf admin admin] PATH_INFO: `/v3/projects/e984738ffdad4015be4aca29cc1a1b7d/users/664fe6d449b7443c925d0713d35eabce/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.148001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86edbc4-8316-49a8-a2f6-1a7e80cfedcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e984738ffdad4015be4aca29cc1a1b7d, user_id=664fe6d449b7443c925d0713d35eabce, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:15.150699 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86edbc4-8316-49a8-a2f6-1a7e80cfedcf admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:15.165077 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1272/2513] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:15 2022] PUT /identity/v3/projects/e984738ffdad4015be4aca29cc1a1b7d/users/664fe6d449b7443c925d0713d35eabce/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:15.176414 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b805b-03d2-48eb-a51d-5dffea941066 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.176707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b805b-03d2-48eb-a51d-5dffea941066 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.176989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b805b-03d2-48eb-a51d-5dffea941066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.205562 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fc6b805b-03d2-48eb-a51d-5dffea941066 None None] MFA Rules not processed for user `664fe6d449b7443c925d0713d35eabce`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:15.206827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fc6b805b-03d2-48eb-a51d-5dffea941066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:15.273325 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1242/2514] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:15.277938 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86edbc4-8316-49a8-a2f6-1a7e80cfedcf admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:15.345505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86edbc4-8316-49a8-a2f6-1a7e80cfedcf admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:15.349996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d8434-3e32-465b-b8a1-3cc0a552ee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:15.350695 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d8434-3e32-465b-b8a1-3cc0a552ee70 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.350981 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d8434-3e32-465b-b8a1-3cc0a552ee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.351255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d8434-3e32-465b-b8a1-3cc0a552ee70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.351962 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d8434-3e32-465b-b8a1-3cc0a552ee70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:15.353859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d8434-3e32-465b-b8a1-3cc0a552ee70 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:15.390396 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1273/2515] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:15 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:18:15.391510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b805b-03d2-48eb-a51d-5dffea941066 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:15.405747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b805b-03d2-48eb-a51d-5dffea941066 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:15.471714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8516bb-3a8f-4fc8-9699-5d3c7ebc88d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:15.472396 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8516bb-3a8f-4fc8-9699-5d3c7ebc88d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.472682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8516bb-3a8f-4fc8-9699-5d3c7ebc88d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.472959 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8516bb-3a8f-4fc8-9699-5d3c7ebc88d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.473652 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8516bb-3a8f-4fc8-9699-5d3c7ebc88d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:15.475711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8516bb-3a8f-4fc8-9699-5d3c7ebc88d4 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:15.498267 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1243/2516] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:15 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:15.500808 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d8434-3e32-465b-b8a1-3cc0a552ee70 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:15.511479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d8434-3e32-465b-b8a1-3cc0a552ee70 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:15.516552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315cd82-0fed-4d7f-b307-b56d7879fccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:15.517308 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315cd82-0fed-4d7f-b307-b56d7879fccc admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.517653 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315cd82-0fed-4d7f-b307-b56d7879fccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.517988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315cd82-0fed-4d7f-b307-b56d7879fccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.518832 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315cd82-0fed-4d7f-b307-b56d7879fccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:15.520618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315cd82-0fed-4d7f-b307-b56d7879fccc admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:15.532095 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1274/2517] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:15.536358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8516bb-3a8f-4fc8-9699-5d3c7ebc88d4 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:15.546746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8516bb-3a8f-4fc8-9699-5d3c7ebc88d4 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:15.552549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d04e1f-a4cd-4cdd-8129-ad3e66b1358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:15.553488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d04e1f-a4cd-4cdd-8129-ad3e66b1358d admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.553767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d04e1f-a4cd-4cdd-8129-ad3e66b1358d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.554060 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d04e1f-a4cd-4cdd-8129-ad3e66b1358d admin admin] PATH_INFO: `/v3/projects/a0ee2ccc83924b0db211a900ef392810/users/df963a91329c40baada0541435c8e866/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.569350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d04e1f-a4cd-4cdd-8129-ad3e66b1358d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ee2ccc83924b0db211a900ef392810, user_id=df963a91329c40baada0541435c8e866, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:15.572072 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d04e1f-a4cd-4cdd-8129-ad3e66b1358d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:15.583183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:15.589627 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1244/2518] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:15 2022] PUT /identity/v3/projects/a0ee2ccc83924b0db211a900ef392810/users/df963a91329c40baada0541435c8e866/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:15.593622 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315cd82-0fed-4d7f-b307-b56d7879fccc admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:15.603958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315cd82-0fed-4d7f-b307-b56d7879fccc admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:15.609368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:15.669214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717d1ba-43de-4abf-abb1-72fe11844c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:15.670162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717d1ba-43de-4abf-abb1-72fe11844c38 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.670726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717d1ba-43de-4abf-abb1-72fe11844c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.671200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717d1ba-43de-4abf-abb1-72fe11844c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.672435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717d1ba-43de-4abf-abb1-72fe11844c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:15.674507 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717d1ba-43de-4abf-abb1-72fe11844c38 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:15.689025 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1275/2519] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:15.691521 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d04e1f-a4cd-4cdd-8129-ad3e66b1358d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:15.703377 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d04e1f-a4cd-4cdd-8129-ad3e66b1358d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:15.709250 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c2284-6fe0-4b86-8668-8e4907b61f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:15.710249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c2284-6fe0-4b86-8668-8e4907b61f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.710612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c2284-6fe0-4b86-8668-8e4907b61f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.710950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c2284-6fe0-4b86-8668-8e4907b61f21 admin admin] PATH_INFO: `/v3/projects/a0ee2ccc83924b0db211a900ef392810/users/df963a91329c40baada0541435c8e866/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.713598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c2284-6fe0-4b86-8668-8e4907b61f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ee2ccc83924b0db211a900ef392810, user_id=df963a91329c40baada0541435c8e866, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:15.716786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c2284-6fe0-4b86-8668-8e4907b61f21 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:15.729754 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1245/2520] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:15 2022] PUT /identity/v3/projects/a0ee2ccc83924b0db211a900ef392810/users/df963a91329c40baada0541435c8e866/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:15.740884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d2d4d-4931-4e44-9bcd-374679b10516 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.741309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d2d4d-4931-4e44-9bcd-374679b10516 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.741693 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d2d4d-4931-4e44-9bcd-374679b10516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.770879 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-b12d2d4d-4931-4e44-9bcd-374679b10516 None None] MFA Rules not processed for user `df963a91329c40baada0541435c8e866`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:15.772106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b12d2d4d-4931-4e44-9bcd-374679b10516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:15.845377 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1276/2521] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:15.853416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c2284-6fe0-4b86-8668-8e4907b61f21 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:15.855248 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fc7c2284-6fe0-4b86-8668-8e4907b61f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:15.911773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c2284-6fe0-4b86-8668-8e4907b61f21 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:15.915606 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f1f16-85fe-4f86-ac51-4897c3d969f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:15.916254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f1f16-85fe-4f86-ac51-4897c3d969f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:15.916539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f1f16-85fe-4f86-ac51-4897c3d969f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:15.916818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f1f16-85fe-4f86-ac51-4897c3d969f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:15.926175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f1f16-85fe-4f86-ac51-4897c3d969f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:15.927634 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f1f16-85fe-4f86-ac51-4897c3d969f9 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:15.937172 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1246/2522] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:16.360557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d2d4d-4931-4e44-9bcd-374679b10516 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:16.421904 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d2d4d-4931-4e44-9bcd-374679b10516 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:16.426522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:16.427407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:16.427701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:16.427975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] PATH_INFO: `/v3/users/664fe6d449b7443c925d0713d35eabce` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:16.429145 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=664fe6d449b7443c925d0713d35eabce)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:16.431091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:16.469736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:16.486617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '664fe6d449b7443c925d0713d35eabce'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:16.487581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '664fe6d449b7443c925d0713d35eabce'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:16.500760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '664fe6d449b7443c925d0713d35eabce'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:16.507834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '664fe6d449b7443c925d0713d35eabce'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:16.508612 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '664fe6d449b7443c925d0713d35eabce'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:16.523686 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1277/2523] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:16 2022] DELETE /identity/v3/users/664fe6d449b7443c925d0713d35eabce => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:18:16.528800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcdb4c-4778-4447-8eef-bf97bb5eebc6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:16.529285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcdb4c-4778-4447-8eef-bf97bb5eebc6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:16.529686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcdb4c-4778-4447-8eef-bf97bb5eebc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:16.550455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5abcdb4c-4778-4447-8eef-bf97bb5eebc6 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:16.551650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5abcdb4c-4778-4447-8eef-bf97bb5eebc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:16.609321 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1247/2524] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:18:16.619432 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:16.621074 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:16.678175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038bf6b-557c-4ceb-973c-0a5c75cf17e9 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:16.682329 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7fe07-d24d-4309-a4aa-ecc0675ef51e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:16.682975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7fe07-d24d-4309-a4aa-ecc0675ef51e service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:16.683257 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7fe07-d24d-4309-a4aa-ecc0675ef51e service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:16.683530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7fe07-d24d-4309-a4aa-ecc0675ef51e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:16.693014 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7fe07-d24d-4309-a4aa-ecc0675ef51e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:16.694513 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7fe07-d24d-4309-a4aa-ecc0675ef51e service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:16.704348 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1278/2525] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:17.063407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abcdb4c-4778-4447-8eef-bf97bb5eebc6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:17.064867 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5abcdb4c-4778-4447-8eef-bf97bb5eebc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:17.099104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abcdb4c-4778-4447-8eef-bf97bb5eebc6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:17.104532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:17.116296 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:17.116801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:17.117260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] PATH_INFO: `/v3/projects/e984738ffdad4015be4aca29cc1a1b7d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:17.124541 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e984738ffdad4015be4aca29cc1a1b7d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:17.126594 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:17.183639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e984738ffdad4015be4aca29cc1a1b7d', 'request_id': 'req-560d0290-ef0d-4abb-8a6a-75e12e4e709e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.194162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e984738ffdad4015be4aca29cc1a1b7d', 'request_id': 'req-560d0290-ef0d-4abb-8a6a-75e12e4e709e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.200709 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e984738ffdad4015be4aca29cc1a1b7d', 'request_id': 'req-560d0290-ef0d-4abb-8a6a-75e12e4e709e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.212277 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] The token cache is being invalidate because project e984738ffdad4015be4aca29cc1a1b7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:17.212507 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.214822 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.217067 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1248/2526] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:17 2022] DELETE /identity/v3/projects/e984738ffdad4015be4aca29cc1a1b7d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:17.224377 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7fe07-d24d-4309-a4aa-ecc0675ef51e service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:17.237363 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7fe07-d24d-4309-a4aa-ecc0675ef51e service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:17.300176 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c646e36-138f-47c0-b0db-77a71c01bc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:17.301076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c646e36-138f-47c0-b0db-77a71c01bc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:17.301357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c646e36-138f-47c0-b0db-77a71c01bc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:17.301638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c646e36-138f-47c0-b0db-77a71c01bc7a admin admin] PATH_INFO: `/v3/users/df963a91329c40baada0541435c8e866` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:17.302950 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c646e36-138f-47c0-b0db-77a71c01bc7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=df963a91329c40baada0541435c8e866)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:17.305002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c646e36-138f-47c0-b0db-77a71c01bc7a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:17.353147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5c646e36-138f-47c0-b0db-77a71c01bc7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df963a91329c40baada0541435c8e866'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.353829 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5c646e36-138f-47c0-b0db-77a71c01bc7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df963a91329c40baada0541435c8e866'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.359164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:17.368078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5c646e36-138f-47c0-b0db-77a71c01bc7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df963a91329c40baada0541435c8e866'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.375673 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5c646e36-138f-47c0-b0db-77a71c01bc7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df963a91329c40baada0541435c8e866'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.376348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-5c646e36-138f-47c0-b0db-77a71c01bc7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df963a91329c40baada0541435c8e866'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.389889 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1279/2527] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:17 2022] DELETE /identity/v3/users/df963a91329c40baada0541435c8e866 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:17.677437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:17.679008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:17.693893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d0290-ef0d-4abb-8a6a-75e12e4e709e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:17.705091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:17.754361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:17.756110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:17.756418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:17.756704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] PATH_INFO: `/v3/projects/a0ee2ccc83924b0db211a900ef392810` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:17.758289 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ee2ccc83924b0db211a900ef392810)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:17.760284 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:17.816728 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ee2ccc83924b0db211a900ef392810', 'request_id': 'req-c71dd991-7b3a-4143-b1da-a5c24e8edd86'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.824843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ee2ccc83924b0db211a900ef392810', 'request_id': 'req-c71dd991-7b3a-4143-b1da-a5c24e8edd86'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.830711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ee2ccc83924b0db211a900ef392810', 'request_id': 'req-c71dd991-7b3a-4143-b1da-a5c24e8edd86'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.842660 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] The token cache is being invalidate because project a0ee2ccc83924b0db211a900ef392810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:17.842948 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.843665 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c71dd991-7b3a-4143-b1da-a5c24e8edd86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:17.845318 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1249/2528] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:17 2022] DELETE /identity/v3/projects/a0ee2ccc83924b0db211a900ef392810 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:17.850444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b649a-7484-46a2-b813-4920a3653362 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:17.850739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b649a-7484-46a2-b813-4920a3653362 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:17.851017 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b649a-7484-46a2-b813-4920a3653362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:17.869487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-bf4b649a-7484-46a2-b813-4920a3653362 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:17.870752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-bf4b649a-7484-46a2-b813-4920a3653362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:17.929110 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1280/2529] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:17.937341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e51268-a53e-4572-97a4-b8005dc6d6fa None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:17.937651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e51268-a53e-4572-97a4-b8005dc6d6fa None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:17.937955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e51268-a53e-4572-97a4-b8005dc6d6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:17.957474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-23e51268-a53e-4572-97a4-b8005dc6d6fa None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:17.958789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-23e51268-a53e-4572-97a4-b8005dc6d6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:17.969075 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1250/2530] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:17.974055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b649a-7484-46a2-b813-4920a3653362 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:17.989193 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b649a-7484-46a2-b813-4920a3653362 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:17.997372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b386488-ed53-4c42-a768-f72a2c77f7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:17.998113 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b386488-ed53-4c42-a768-f72a2c77f7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:17.998430 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b386488-ed53-4c42-a768-f72a2c77f7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:17.998706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b386488-ed53-4c42-a768-f72a2c77f7b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:17.999435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b386488-ed53-4c42-a768-f72a2c77f7b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:18:17.999741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b386488-ed53-4c42-a768-f72a2c77f7b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.001367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b386488-ed53-4c42-a768-f72a2c77f7b4 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.012958 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1281/2531] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:18:18.017781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e51268-a53e-4572-97a4-b8005dc6d6fa None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.031247 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e51268-a53e-4572-97a4-b8005dc6d6fa None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:18.037028 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79b3e9-d438-40c0-b036-35b32dd697c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:18.037698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79b3e9-d438-40c0-b036-35b32dd697c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.037981 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79b3e9-d438-40c0-b036-35b32dd697c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.038261 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79b3e9-d438-40c0-b036-35b32dd697c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.039003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79b3e9-d438-40c0-b036-35b32dd697c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.040894 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79b3e9-d438-40c0-b036-35b32dd697c3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.069396 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1251/2532] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:18 2022] POST /identity/v3/projects => generated 355 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:18.073631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b386488-ed53-4c42-a768-f72a2c77f7b4 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.084063 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b386488-ed53-4c42-a768-f72a2c77f7b4 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:18.140780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0ba64-714a-436d-bfa3-8526b25da5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:18.141461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0ba64-714a-436d-bfa3-8526b25da5fa admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.141803 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0ba64-714a-436d-bfa3-8526b25da5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.142071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0ba64-714a-436d-bfa3-8526b25da5fa admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.142808 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0ba64-714a-436d-bfa3-8526b25da5fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.144778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0ba64-714a-436d-bfa3-8526b25da5fa admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.165503 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1282/2533] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:18 2022] POST /identity/v3/users => generated 278 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:18.169856 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79b3e9-d438-40c0-b036-35b32dd697c3 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.180628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79b3e9-d438-40c0-b036-35b32dd697c3 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:18.186140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221750b-6d68-441b-b8d6-c47a22f5e24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:18.187004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221750b-6d68-441b-b8d6-c47a22f5e24c admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.187348 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221750b-6d68-441b-b8d6-c47a22f5e24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.187694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221750b-6d68-441b-b8d6-c47a22f5e24c admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.188494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221750b-6d68-441b-b8d6-c47a22f5e24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.190162 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221750b-6d68-441b-b8d6-c47a22f5e24c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.201587 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1252/2534] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:18:18.205930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0ba64-714a-436d-bfa3-8526b25da5fa admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.217881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0ba64-714a-436d-bfa3-8526b25da5fa admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:18.223971 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cc8f5-06bb-4879-81d4-6ad9a98d7578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:18.224742 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cc8f5-06bb-4879-81d4-6ad9a98d7578 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.225313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cc8f5-06bb-4879-81d4-6ad9a98d7578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.225313 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cc8f5-06bb-4879-81d4-6ad9a98d7578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.226014 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cc8f5-06bb-4879-81d4-6ad9a98d7578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.227736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cc8f5-06bb-4879-81d4-6ad9a98d7578 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.242367 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1283/2535] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:18.244048 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221750b-6d68-441b-b8d6-c47a22f5e24c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.254526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221750b-6d68-441b-b8d6-c47a22f5e24c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:18.260616 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add4b63-e3a3-4361-881e-b6c57ab44a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:18.261531 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add4b63-e3a3-4361-881e-b6c57ab44a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.261995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add4b63-e3a3-4361-881e-b6c57ab44a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.262366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add4b63-e3a3-4361-881e-b6c57ab44a22 admin admin] PATH_INFO: `/v3/projects/0d7d7c43568d455db252dc6321976bd4/users/1f689d5b7ab340e7973e8e722c13ebdb/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.280142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add4b63-e3a3-4361-881e-b6c57ab44a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d7d7c43568d455db252dc6321976bd4, user_id=1f689d5b7ab340e7973e8e722c13ebdb, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.282851 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add4b63-e3a3-4361-881e-b6c57ab44a22 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.297708 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1253/2536] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:18 2022] PUT /identity/v3/projects/0d7d7c43568d455db252dc6321976bd4/users/1f689d5b7ab340e7973e8e722c13ebdb/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:18.308972 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a888d3-3f57-4d28-9031-59bc611f5b6e None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.309277 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a888d3-3f57-4d28-9031-59bc611f5b6e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.309584 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a888d3-3f57-4d28-9031-59bc611f5b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.336289 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-53a888d3-3f57-4d28-9031-59bc611f5b6e None None] MFA Rules not processed for user `1f689d5b7ab340e7973e8e722c13ebdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:18.337504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-53a888d3-3f57-4d28-9031-59bc611f5b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:18.363891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:18.414846 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1284/2537] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:18 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:18.419786 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add4b63-e3a3-4361-881e-b6c57ab44a22 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.470809 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:18.496894 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add4b63-e3a3-4361-881e-b6c57ab44a22 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:18.501713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77c129-5147-4ac7-9cf5-83e2b1a10834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:18.502422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77c129-5147-4ac7-9cf5-83e2b1a10834 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.502707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77c129-5147-4ac7-9cf5-83e2b1a10834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.502985 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77c129-5147-4ac7-9cf5-83e2b1a10834 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.503701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77c129-5147-4ac7-9cf5-83e2b1a10834 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.505628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77c129-5147-4ac7-9cf5-83e2b1a10834 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.546506 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1254/2538] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:18 2022] POST /identity/v3/projects => generated 357 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:18.547749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a888d3-3f57-4d28-9031-59bc611f5b6e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.560255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a888d3-3f57-4d28-9031-59bc611f5b6e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:18.617996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2490f3-1b41-4442-a7b4-23a5d1bf102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:18.618718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2490f3-1b41-4442-a7b4-23a5d1bf102c admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.619000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2490f3-1b41-4442-a7b4-23a5d1bf102c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.619272 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2490f3-1b41-4442-a7b4-23a5d1bf102c admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.619922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2490f3-1b41-4442-a7b4-23a5d1bf102c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.621938 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2490f3-1b41-4442-a7b4-23a5d1bf102c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.644437 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1285/2539] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:18 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:18.648044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77c129-5147-4ac7-9cf5-83e2b1a10834 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.659756 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77c129-5147-4ac7-9cf5-83e2b1a10834 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:18.665236 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371facaa-f0a4-484b-a79f-31cbf9aa70fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:18.666034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371facaa-f0a4-484b-a79f-31cbf9aa70fe admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.666354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371facaa-f0a4-484b-a79f-31cbf9aa70fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.666628 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371facaa-f0a4-484b-a79f-31cbf9aa70fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.667382 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371facaa-f0a4-484b-a79f-31cbf9aa70fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.668999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371facaa-f0a4-484b-a79f-31cbf9aa70fe admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.680712 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1255/2540] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:18.684985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2490f3-1b41-4442-a7b4-23a5d1bf102c admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.696762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2490f3-1b41-4442-a7b4-23a5d1bf102c admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:18.701508 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01462d1-8d44-44da-a72f-255685745e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:18.702662 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01462d1-8d44-44da-a72f-255685745e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.703170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01462d1-8d44-44da-a72f-255685745e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.703516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01462d1-8d44-44da-a72f-255685745e2f admin admin] PATH_INFO: `/v3/projects/28fa16f6e3c248a8847a871d8479f30a/users/b0936789b23d492abcccf0335ec82b6e/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.724384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01462d1-8d44-44da-a72f-255685745e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=28fa16f6e3c248a8847a871d8479f30a, user_id=b0936789b23d492abcccf0335ec82b6e, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.727229 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01462d1-8d44-44da-a72f-255685745e2f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.743266 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1286/2541] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:18 2022] PUT /identity/v3/projects/28fa16f6e3c248a8847a871d8479f30a/users/b0936789b23d492abcccf0335ec82b6e/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:18.751483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371facaa-f0a4-484b-a79f-31cbf9aa70fe admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.758605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371facaa-f0a4-484b-a79f-31cbf9aa70fe admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:18.818571 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259404e-3a95-4f36-a4e4-fff8afd20203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:18.819248 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259404e-3a95-4f36-a4e4-fff8afd20203 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.819530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259404e-3a95-4f36-a4e4-fff8afd20203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.819818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259404e-3a95-4f36-a4e4-fff8afd20203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.820544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259404e-3a95-4f36-a4e4-fff8afd20203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.822136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259404e-3a95-4f36-a4e4-fff8afd20203 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.832535 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1256/2542] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:18.836801 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01462d1-8d44-44da-a72f-255685745e2f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.847415 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01462d1-8d44-44da-a72f-255685745e2f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:18.852102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc1f79-5a27-4679-9056-558f5d7ba1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:18.852997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc1f79-5a27-4679-9056-558f5d7ba1f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.853280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc1f79-5a27-4679-9056-558f5d7ba1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.853554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc1f79-5a27-4679-9056-558f5d7ba1f2 admin admin] PATH_INFO: `/v3/projects/28fa16f6e3c248a8847a871d8479f30a/users/b0936789b23d492abcccf0335ec82b6e/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.855594 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc1f79-5a27-4679-9056-558f5d7ba1f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28fa16f6e3c248a8847a871d8479f30a, user_id=b0936789b23d492abcccf0335ec82b6e, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:18.858294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc1f79-5a27-4679-9056-558f5d7ba1f2 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:18.869449 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1287/2543] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:18 2022] PUT /identity/v3/projects/28fa16f6e3c248a8847a871d8479f30a/users/b0936789b23d492abcccf0335ec82b6e/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:18.880023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbc4b1-7afc-4e49-9ffa-46f2781dd1e3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:18.880309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbc4b1-7afc-4e49-9ffa-46f2781dd1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:18.880584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbc4b1-7afc-4e49-9ffa-46f2781dd1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:18.908411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-17dbc4b1-7afc-4e49-9ffa-46f2781dd1e3 None None] MFA Rules not processed for user `b0936789b23d492abcccf0335ec82b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:18.909629 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-17dbc4b1-7afc-4e49-9ffa-46f2781dd1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:18.985532 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1257/2544] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:18 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:18.993299 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc1f79-5a27-4679-9056-558f5d7ba1f2 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:18.995460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5cfc1f79-5a27-4679-9056-558f5d7ba1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:19.056317 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc1f79-5a27-4679-9056-558f5d7ba1f2 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:19.060654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363067e1-0929-414a-9b9f-70d64e82353c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:19.061341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363067e1-0929-414a-9b9f-70d64e82353c service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:19.061628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363067e1-0929-414a-9b9f-70d64e82353c service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:19.061906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363067e1-0929-414a-9b9f-70d64e82353c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:19.071691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363067e1-0929-414a-9b9f-70d64e82353c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:19.073173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363067e1-0929-414a-9b9f-70d64e82353c service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:19.083200 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1288/2545] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:18 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:19.157816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbc4b1-7afc-4e49-9ffa-46f2781dd1e3 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:19.168163 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbc4b1-7afc-4e49-9ffa-46f2781dd1e3 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:19.172293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ef7cf-ba4f-4dc7-817f-7f242655961a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:19.172980 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ef7cf-ba4f-4dc7-817f-7f242655961a service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:19.173219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ef7cf-ba4f-4dc7-817f-7f242655961a service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:19.173493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ef7cf-ba4f-4dc7-817f-7f242655961a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:19.208089 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:19.235443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ef7cf-ba4f-4dc7-817f-7f242655961a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:19.237336 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ef7cf-ba4f-4dc7-817f-7f242655961a service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:19.247679 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1258/2546] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:19 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:29.078612 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=85,keystone:INSERT=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:29.249989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=12 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:34.329841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363067e1-0929-414a-9b9f-70d64e82353c service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:34.392381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363067e1-0929-414a-9b9f-70d64e82353c service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:34.398500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:34.399413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:34.399692 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:34.399965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] PATH_INFO: `/v3/users/1f689d5b7ab340e7973e8e722c13ebdb` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:34.401383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f689d5b7ab340e7973e8e722c13ebdb)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:34.403417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:34.452828 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f689d5b7ab340e7973e8e722c13ebdb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:34.453521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f689d5b7ab340e7973e8e722c13ebdb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:34.466334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f689d5b7ab340e7973e8e722c13ebdb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:34.473606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f689d5b7ab340e7973e8e722c13ebdb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:34.474274 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f689d5b7ab340e7973e8e722c13ebdb'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:34.489088 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1289/2547] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:34 2022] DELETE /identity/v3/users/1f689d5b7ab340e7973e8e722c13ebdb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:34.493667 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28e984-32b8-48c2-a9fa-d9e83df50618 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:34.494140 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28e984-32b8-48c2-a9fa-d9e83df50618 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:34.494288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28e984-32b8-48c2-a9fa-d9e83df50618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:34.513164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dc28e984-32b8-48c2-a9fa-d9e83df50618 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:34.514461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dc28e984-32b8-48c2-a9fa-d9e83df50618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:34.572730 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1259/2548] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:34.584960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:34.587277 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:34.648623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161708e-dfa8-48e7-82c8-7f6abffe5cd0 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:34.652333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bbd63-58a9-4b61-adbb-d7913b23d882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:34.652977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bbd63-58a9-4b61-adbb-d7913b23d882 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:34.653256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bbd63-58a9-4b61-adbb-d7913b23d882 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:34.653528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bbd63-58a9-4b61-adbb-d7913b23d882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:34.662742 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bbd63-58a9-4b61-adbb-d7913b23d882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:34.664189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bbd63-58a9-4b61-adbb-d7913b23d882 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:34.673652 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1290/2549] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:34.951609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28e984-32b8-48c2-a9fa-d9e83df50618 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:34.953832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dc28e984-32b8-48c2-a9fa-d9e83df50618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:34.967795 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28e984-32b8-48c2-a9fa-d9e83df50618 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:34.973180 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:34.974067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:34.974514 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:34.974877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] PATH_INFO: `/v3/projects/0d7d7c43568d455db252dc6321976bd4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:34.976049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d7d7c43568d455db252dc6321976bd4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:34.981105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:35.034650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d7d7c43568d455db252dc6321976bd4', 'request_id': 'req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.043729 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d7d7c43568d455db252dc6321976bd4', 'request_id': 'req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.049697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d7d7c43568d455db252dc6321976bd4', 'request_id': 'req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.061113 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] The token cache is being invalidate because project 0d7d7c43568d455db252dc6321976bd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:35.061400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.062092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.063738 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1260/2550] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:34 2022] DELETE /identity/v3/projects/0d7d7c43568d455db252dc6321976bd4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:35.067949 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bbd63-58a9-4b61-adbb-d7913b23d882 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:35.078021 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bbd63-58a9-4b61-adbb-d7913b23d882 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:35.098636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:35.145220 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73294e5-2d6b-40f5-a49f-2f400c4805ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:35.146102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73294e5-2d6b-40f5-a49f-2f400c4805ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:35.146453 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73294e5-2d6b-40f5-a49f-2f400c4805ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:35.146724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73294e5-2d6b-40f5-a49f-2f400c4805ec admin admin] PATH_INFO: `/v3/users/b0936789b23d492abcccf0335ec82b6e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:35.147939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73294e5-2d6b-40f5-a49f-2f400c4805ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0936789b23d492abcccf0335ec82b6e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:35.149875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73294e5-2d6b-40f5-a49f-2f400c4805ec admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:35.196932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b73294e5-2d6b-40f5-a49f-2f400c4805ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0936789b23d492abcccf0335ec82b6e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.197566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b73294e5-2d6b-40f5-a49f-2f400c4805ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0936789b23d492abcccf0335ec82b6e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.209179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b73294e5-2d6b-40f5-a49f-2f400c4805ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0936789b23d492abcccf0335ec82b6e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.216643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b73294e5-2d6b-40f5-a49f-2f400c4805ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0936789b23d492abcccf0335ec82b6e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.217304 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b73294e5-2d6b-40f5-a49f-2f400c4805ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0936789b23d492abcccf0335ec82b6e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.231871 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1291/2551] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:35 2022] DELETE /identity/v3/users/b0936789b23d492abcccf0335ec82b6e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:35.515006 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:35.516692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:35.530876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9aa6b-38a1-4778-8775-165b4f3a0b63 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:35.590537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:35.591398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:35.591675 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:35.591961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] PATH_INFO: `/v3/projects/28fa16f6e3c248a8847a871d8479f30a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:35.593262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=28fa16f6e3c248a8847a871d8479f30a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:35.595219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:35.627898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:35.646096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28fa16f6e3c248a8847a871d8479f30a', 'request_id': 'req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.653944 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28fa16f6e3c248a8847a871d8479f30a', 'request_id': 'req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.659496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28fa16f6e3c248a8847a871d8479f30a', 'request_id': 'req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.670430 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] The token cache is being invalidate because project 28fa16f6e3c248a8847a871d8479f30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:35.670717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.671343 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-40d5f5fc-f3b4-4726-bca3-06a91e578d7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:35.672883 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1261/2552] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:35 2022] DELETE /identity/v3/projects/28fa16f6e3c248a8847a871d8479f30a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:35.678085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee293a0-bf4a-48c8-9fd7-712cfa64d1d9 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:35.678640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee293a0-bf4a-48c8-9fd7-712cfa64d1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:35.678738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee293a0-bf4a-48c8-9fd7-712cfa64d1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:35.697059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-5ee293a0-bf4a-48c8-9fd7-712cfa64d1d9 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:35.698271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5ee293a0-bf4a-48c8-9fd7-712cfa64d1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:35.762310 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1292/2553] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:35.768157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1dc7e-fba8-4854-bfac-514721863ca9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:35.768450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1dc7e-fba8-4854-bfac-514721863ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:35.768720 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1dc7e-fba8-4854-bfac-514721863ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:35.788256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a4e1dc7e-fba8-4854-bfac-514721863ca9 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:35.789453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a4e1dc7e-fba8-4854-bfac-514721863ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:35.798520 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1262/2554] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:35.803523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee293a0-bf4a-48c8-9fd7-712cfa64d1d9 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:35.820053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee293a0-bf4a-48c8-9fd7-712cfa64d1d9 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:35.825480 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ee430-c3f1-4455-b0ea-cd222224bfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:35.826185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ee430-c3f1-4455-b0ea-cd222224bfae admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:35.826493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ee430-c3f1-4455-b0ea-cd222224bfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:35.826783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ee430-c3f1-4455-b0ea-cd222224bfae admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:35.827493 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ee430-c3f1-4455-b0ea-cd222224bfae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:18:35.827799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ee430-c3f1-4455-b0ea-cd222224bfae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:35.829445 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ee430-c3f1-4455-b0ea-cd222224bfae admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:35.841644 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1293/2555] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:18:35.846185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1dc7e-fba8-4854-bfac-514721863ca9 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:35.855985 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1dc7e-fba8-4854-bfac-514721863ca9 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:35.861770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f3fc7-a7f3-4deb-8011-473e5624cd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:35.862468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f3fc7-a7f3-4deb-8011-473e5624cd71 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:35.862752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f3fc7-a7f3-4deb-8011-473e5624cd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:35.863044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f3fc7-a7f3-4deb-8011-473e5624cd71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:35.863731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f3fc7-a7f3-4deb-8011-473e5624cd71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:35.865627 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f3fc7-a7f3-4deb-8011-473e5624cd71 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:35.906884 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1263/2556] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:35 2022] POST /identity/v3/projects => generated 363 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:35.912306 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ee430-c3f1-4455-b0ea-cd222224bfae admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:35.922812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ee430-c3f1-4455-b0ea-cd222224bfae admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:35.948841 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:35.981801 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16fa66-4bf0-47a2-8754-ab557154d0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:35.982529 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16fa66-4bf0-47a2-8754-ab557154d0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:35.982817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16fa66-4bf0-47a2-8754-ab557154d0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:35.983091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16fa66-4bf0-47a2-8754-ab557154d0a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:35.983782 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16fa66-4bf0-47a2-8754-ab557154d0a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:35.985744 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16fa66-4bf0-47a2-8754-ab557154d0a6 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:36.013974 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1294/2557] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:35 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:36.018313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f3fc7-a7f3-4deb-8011-473e5624cd71 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:36.029255 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f3fc7-a7f3-4deb-8011-473e5624cd71 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:36.033974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb166f-30e8-4fbf-80b2-08bcf951bf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:36.034670 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb166f-30e8-4fbf-80b2-08bcf951bf8f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.034958 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb166f-30e8-4fbf-80b2-08bcf951bf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.035234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb166f-30e8-4fbf-80b2-08bcf951bf8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.036004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb166f-30e8-4fbf-80b2-08bcf951bf8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:36.037605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb166f-30e8-4fbf-80b2-08bcf951bf8f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:36.047543 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1264/2558] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:36.051691 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16fa66-4bf0-47a2-8754-ab557154d0a6 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:36.062110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16fa66-4bf0-47a2-8754-ab557154d0a6 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:36.067500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f2289-5044-4e8f-8995-4ceb4dd963db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:36.068169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f2289-5044-4e8f-8995-4ceb4dd963db admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.068454 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f2289-5044-4e8f-8995-4ceb4dd963db admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.068733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f2289-5044-4e8f-8995-4ceb4dd963db admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.069461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f2289-5044-4e8f-8995-4ceb4dd963db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:36.071170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f2289-5044-4e8f-8995-4ceb4dd963db admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:36.081396 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1295/2559] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:36.085369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb166f-30e8-4fbf-80b2-08bcf951bf8f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:36.096017 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb166f-30e8-4fbf-80b2-08bcf951bf8f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:36.101207 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e375c5-085b-4033-a12a-fd27eb205483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:36.102087 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e375c5-085b-4033-a12a-fd27eb205483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.102406 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e375c5-085b-4033-a12a-fd27eb205483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.102686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e375c5-085b-4033-a12a-fd27eb205483 admin admin] PATH_INFO: `/v3/projects/c61ebd82ab0c4350b0b80dadbe8b86ff/users/18a954427e37496da31d539a7414fe7f/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.117695 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e375c5-085b-4033-a12a-fd27eb205483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61ebd82ab0c4350b0b80dadbe8b86ff, user_id=18a954427e37496da31d539a7414fe7f, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:36.120343 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e375c5-085b-4033-a12a-fd27eb205483 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:36.136928 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1265/2560] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:36 2022] PUT /identity/v3/projects/c61ebd82ab0c4350b0b80dadbe8b86ff/users/18a954427e37496da31d539a7414fe7f/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:36.147243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974fd07-2bcc-4f1b-ab31-86beb3bc4ea7 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.147531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974fd07-2bcc-4f1b-ab31-86beb3bc4ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.147811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974fd07-2bcc-4f1b-ab31-86beb3bc4ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.174282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-2974fd07-2bcc-4f1b-ab31-86beb3bc4ea7 None None] MFA Rules not processed for user `18a954427e37496da31d539a7414fe7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:36.175407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2974fd07-2bcc-4f1b-ab31-86beb3bc4ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:36.241513 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1296/2561] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:36 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:36.246104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e375c5-085b-4033-a12a-fd27eb205483 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:36.308908 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e375c5-085b-4033-a12a-fd27eb205483 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:36.314186 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b1aab-5b12-4588-850a-cd83b43a9cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:36.314870 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b1aab-5b12-4588-850a-cd83b43a9cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.315155 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b1aab-5b12-4588-850a-cd83b43a9cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.315430 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b1aab-5b12-4588-850a-cd83b43a9cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.316110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b1aab-5b12-4588-850a-cd83b43a9cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:36.318034 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b1aab-5b12-4588-850a-cd83b43a9cb6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:36.346926 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1266/2562] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:36 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:36.351074 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974fd07-2bcc-4f1b-ab31-86beb3bc4ea7 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:36.362642 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974fd07-2bcc-4f1b-ab31-86beb3bc4ea7 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:36.422068 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f20f2c-fe1a-401b-9597-45cab81eea69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:36.422774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f20f2c-fe1a-401b-9597-45cab81eea69 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.423050 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f20f2c-fe1a-401b-9597-45cab81eea69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.423326 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f20f2c-fe1a-401b-9597-45cab81eea69 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.424001 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f20f2c-fe1a-401b-9597-45cab81eea69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:36.426082 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f20f2c-fe1a-401b-9597-45cab81eea69 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:36.445413 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1297/2563] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:36 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:36.449518 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b1aab-5b12-4588-850a-cd83b43a9cb6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:36.460097 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b1aab-5b12-4588-850a-cd83b43a9cb6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:36.465420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c4a4d-ab37-411b-aae4-77edd2677fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:36.466104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c4a4d-ab37-411b-aae4-77edd2677fec admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.466418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c4a4d-ab37-411b-aae4-77edd2677fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.466691 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c4a4d-ab37-411b-aae4-77edd2677fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.467438 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c4a4d-ab37-411b-aae4-77edd2677fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:36.469077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c4a4d-ab37-411b-aae4-77edd2677fec admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:36.480099 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1267/2564] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:36.484502 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f20f2c-fe1a-401b-9597-45cab81eea69 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:36.498858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f20f2c-fe1a-401b-9597-45cab81eea69 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:36.504815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463be1fa-745f-4be8-a649-7d802b466f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:36.505745 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463be1fa-745f-4be8-a649-7d802b466f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.506053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463be1fa-745f-4be8-a649-7d802b466f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.506305 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463be1fa-745f-4be8-a649-7d802b466f2b admin admin] PATH_INFO: `/v3/projects/0774f8a5a30b446ca06105fc7a2c004d/users/0189e872c6ae444ebe14f87c83ce233e/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.523542 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463be1fa-745f-4be8-a649-7d802b466f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0774f8a5a30b446ca06105fc7a2c004d, user_id=0189e872c6ae444ebe14f87c83ce233e, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:36.527683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463be1fa-745f-4be8-a649-7d802b466f2b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:36.544517 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1298/2565] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:36 2022] PUT /identity/v3/projects/0774f8a5a30b446ca06105fc7a2c004d/users/0189e872c6ae444ebe14f87c83ce233e/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:36.548761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c4a4d-ab37-411b-aae4-77edd2677fec admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:36.559233 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c4a4d-ab37-411b-aae4-77edd2677fec admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:36.571734 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:36.619145 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b658e39-5d8a-4cf6-9d18-c51530223f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:36.619816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b658e39-5d8a-4cf6-9d18-c51530223f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.620117 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b658e39-5d8a-4cf6-9d18-c51530223f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.620389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b658e39-5d8a-4cf6-9d18-c51530223f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.621115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b658e39-5d8a-4cf6-9d18-c51530223f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:36.622774 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b658e39-5d8a-4cf6-9d18-c51530223f82 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:36.633282 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1268/2566] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:36.637403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463be1fa-745f-4be8-a649-7d802b466f2b admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:36.643283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:36.650468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463be1fa-745f-4be8-a649-7d802b466f2b admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:36.655415 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b815be-e772-4571-9ad9-f286319e501d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:36.656301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b815be-e772-4571-9ad9-f286319e501d admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.656573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b815be-e772-4571-9ad9-f286319e501d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.656868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b815be-e772-4571-9ad9-f286319e501d admin admin] PATH_INFO: `/v3/projects/0774f8a5a30b446ca06105fc7a2c004d/users/0189e872c6ae444ebe14f87c83ce233e/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.658888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b815be-e772-4571-9ad9-f286319e501d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0774f8a5a30b446ca06105fc7a2c004d, user_id=0189e872c6ae444ebe14f87c83ce233e, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:36.661608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b815be-e772-4571-9ad9-f286319e501d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:36.671793 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1299/2567] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:36 2022] PUT /identity/v3/projects/0774f8a5a30b446ca06105fc7a2c004d/users/0189e872c6ae444ebe14f87c83ce233e/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:36.682074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b49ea-a9ad-417b-a5b1-4a0f7a5f6e46 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.682339 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b49ea-a9ad-417b-a5b1-4a0f7a5f6e46 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.682607 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b49ea-a9ad-417b-a5b1-4a0f7a5f6e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.709700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7a1b49ea-a9ad-417b-a5b1-4a0f7a5f6e46 None None] MFA Rules not processed for user `0189e872c6ae444ebe14f87c83ce233e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:36.710960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7a1b49ea-a9ad-417b-a5b1-4a0f7a5f6e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:36.790628 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1269/2568] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:36.808107 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b815be-e772-4571-9ad9-f286319e501d admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:36.811093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e9b815be-e772-4571-9ad9-f286319e501d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:36.879170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b815be-e772-4571-9ad9-f286319e501d admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:36.884195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a01f1-32c8-453f-b007-a41a6d8ad918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:36.884899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a01f1-32c8-453f-b007-a41a6d8ad918 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:36.885181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a01f1-32c8-453f-b007-a41a6d8ad918 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:36.885452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a01f1-32c8-453f-b007-a41a6d8ad918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:36.896238 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a01f1-32c8-453f-b007-a41a6d8ad918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:36.897681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a01f1-32c8-453f-b007-a41a6d8ad918 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:36.907521 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1300/2569] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:36.988481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b49ea-a9ad-417b-a5b1-4a0f7a5f6e46 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:36.999276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b49ea-a9ad-417b-a5b1-4a0f7a5f6e46 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:37.003797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18d9e4-a9a6-474b-9935-1839dff7dd08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:37.004512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18d9e4-a9a6-474b-9935-1839dff7dd08 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:37.004796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18d9e4-a9a6-474b-9935-1839dff7dd08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:37.005057 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18d9e4-a9a6-474b-9935-1839dff7dd08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:37.064361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18d9e4-a9a6-474b-9935-1839dff7dd08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:37.065812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18d9e4-a9a6-474b-9935-1839dff7dd08 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:37.074838 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1270/2570] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:36 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:39.841774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a01f1-32c8-453f-b007-a41a6d8ad918 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:39.910541 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a01f1-32c8-453f-b007-a41a6d8ad918 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:39.915770 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:39.916712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:39.916993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:39.917269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] PATH_INFO: `/v3/users/18a954427e37496da31d539a7414fe7f` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:39.918559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a954427e37496da31d539a7414fe7f)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:39.920500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:39.971045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a954427e37496da31d539a7414fe7f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:39.971604 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a954427e37496da31d539a7414fe7f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:39.983006 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a954427e37496da31d539a7414fe7f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:39.989540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a954427e37496da31d539a7414fe7f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:39.990196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a954427e37496da31d539a7414fe7f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:40.004543 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1301/2571] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:39 2022] DELETE /identity/v3/users/18a954427e37496da31d539a7414fe7f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:40.009252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c49d7f-3ac3-45ae-a685-3ed4ee247312 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:40.009759 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c49d7f-3ac3-45ae-a685-3ed4ee247312 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:40.009873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c49d7f-3ac3-45ae-a685-3ed4ee247312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:40.028174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-a1c49d7f-3ac3-45ae-a685-3ed4ee247312 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:40.029399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a1c49d7f-3ac3-45ae-a685-3ed4ee247312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:40.059503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:40.091082 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1271/2572] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:40.103662 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:40.105410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:40.120501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:40.165978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f60b1-9606-4d81-ac63-0cb8f38a995f admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:40.169965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275213c-55f9-463f-b979-d04c33b99697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:40.170643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275213c-55f9-463f-b979-d04c33b99697 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:40.170920 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275213c-55f9-463f-b979-d04c33b99697 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:40.171198 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275213c-55f9-463f-b979-d04c33b99697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:40.180566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275213c-55f9-463f-b979-d04c33b99697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:40.181998 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275213c-55f9-463f-b979-d04c33b99697 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:40.191600 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1302/2573] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:40.469834 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c49d7f-3ac3-45ae-a685-3ed4ee247312 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:40.471463 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a1c49d7f-3ac3-45ae-a685-3ed4ee247312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:40.484391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c49d7f-3ac3-45ae-a685-3ed4ee247312 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:40.489997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:40.490927 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:40.491208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:40.491491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] PATH_INFO: `/v3/projects/c61ebd82ab0c4350b0b80dadbe8b86ff` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:40.492854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61ebd82ab0c4350b0b80dadbe8b86ff)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:40.494787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:40.544125 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61ebd82ab0c4350b0b80dadbe8b86ff', 'request_id': 'req-3251dc3b-1a7a-4797-8143-2d4db3256b5d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:40.552482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61ebd82ab0c4350b0b80dadbe8b86ff', 'request_id': 'req-3251dc3b-1a7a-4797-8143-2d4db3256b5d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:40.557836 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61ebd82ab0c4350b0b80dadbe8b86ff', 'request_id': 'req-3251dc3b-1a7a-4797-8143-2d4db3256b5d'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:40.569749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] The token cache is being invalidate because project c61ebd82ab0c4350b0b80dadbe8b86ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:40.570049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:40.570830 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:40.572591 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1272/2574] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:40 2022] DELETE /identity/v3/projects/c61ebd82ab0c4350b0b80dadbe8b86ff => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:40.576865 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275213c-55f9-463f-b979-d04c33b99697 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:40.587029 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275213c-55f9-463f-b979-d04c33b99697 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:40.648290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b93be-85e1-4282-b948-f1addb1afed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:40.649345 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b93be-85e1-4282-b948-f1addb1afed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:40.649658 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b93be-85e1-4282-b948-f1addb1afed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:40.649913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b93be-85e1-4282-b948-f1addb1afed5 admin admin] PATH_INFO: `/v3/users/0189e872c6ae444ebe14f87c83ce233e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:40.651178 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b93be-85e1-4282-b948-f1addb1afed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0189e872c6ae444ebe14f87c83ce233e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:40.653154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b93be-85e1-4282-b948-f1addb1afed5 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:40.701339 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-047b93be-85e1-4282-b948-f1addb1afed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0189e872c6ae444ebe14f87c83ce233e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:40.702020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-047b93be-85e1-4282-b948-f1addb1afed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0189e872c6ae444ebe14f87c83ce233e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:40.714377 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-047b93be-85e1-4282-b948-f1addb1afed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0189e872c6ae444ebe14f87c83ce233e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:40.721806 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-047b93be-85e1-4282-b948-f1addb1afed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0189e872c6ae444ebe14f87c83ce233e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:40.722583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-047b93be-85e1-4282-b948-f1addb1afed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0189e872c6ae444ebe14f87c83ce233e'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:40.736885 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1303/2575] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:40 2022] DELETE /identity/v3/users/0189e872c6ae444ebe14f87c83ce233e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:41.027743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:41.029542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:41.043956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251dc3b-1a7a-4797-8143-2d4db3256b5d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:41.101805 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:41.102617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.102900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.103175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] PATH_INFO: `/v3/projects/0774f8a5a30b446ca06105fc7a2c004d` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.104377 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0774f8a5a30b446ca06105fc7a2c004d)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:41.106327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:41.152999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0774f8a5a30b446ca06105fc7a2c004d', 'request_id': 'req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:41.161767 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0774f8a5a30b446ca06105fc7a2c004d', 'request_id': 'req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:41.165823 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:41.168064 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0774f8a5a30b446ca06105fc7a2c004d', 'request_id': 'req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:41.180399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] The token cache is being invalidate because project 0774f8a5a30b446ca06105fc7a2c004d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:41.180682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:41.181297 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-c8e40fc5-c20d-476f-84eb-a1da0c8320c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:41.182863 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1273/2576] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:41 2022] DELETE /identity/v3/projects/0774f8a5a30b446ca06105fc7a2c004d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:41.188052 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c154d-3edd-473a-bd9e-1de39d9062c7 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.188559 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c154d-3edd-473a-bd9e-1de39d9062c7 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.188669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c154d-3edd-473a-bd9e-1de39d9062c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.209007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-d91c154d-3edd-473a-bd9e-1de39d9062c7 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:41.210258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d91c154d-3edd-473a-bd9e-1de39d9062c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:41.244102 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:41.276669 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1304/2577] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:41.285866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd386-809d-40ee-aae8-19886542b7b4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.286152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd386-809d-40ee-aae8-19886542b7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.286455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd386-809d-40ee-aae8-19886542b7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.306244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fd1fd386-809d-40ee-aae8-19886542b7b4 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:41.307449 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fd1fd386-809d-40ee-aae8-19886542b7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:41.316704 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1274/2578] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:41.321255 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c154d-3edd-473a-bd9e-1de39d9062c7 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:41.331348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c154d-3edd-473a-bd9e-1de39d9062c7 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:41.336622 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401de8b-9d29-4e66-8a69-a27b5072dff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:41.337307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401de8b-9d29-4e66-8a69-a27b5072dff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.337586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401de8b-9d29-4e66-8a69-a27b5072dff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.337839 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401de8b-9d29-4e66-8a69-a27b5072dff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.338930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401de8b-9d29-4e66-8a69-a27b5072dff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:18:41.339235 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401de8b-9d29-4e66-8a69-a27b5072dff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:41.340836 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401de8b-9d29-4e66-8a69-a27b5072dff0 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:41.352445 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1305/2579] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:18:41.356656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd386-809d-40ee-aae8-19886542b7b4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:41.366172 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd386-809d-40ee-aae8-19886542b7b4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:41.371261 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12427b-75db-491e-9047-0672d7e21678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:41.371951 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12427b-75db-491e-9047-0672d7e21678 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.372233 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12427b-75db-491e-9047-0672d7e21678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.372529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12427b-75db-491e-9047-0672d7e21678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.373213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12427b-75db-491e-9047-0672d7e21678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:41.375177 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12427b-75db-491e-9047-0672d7e21678 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:41.402196 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1275/2580] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:41 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:18:41.406505 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401de8b-9d29-4e66-8a69-a27b5072dff0 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:41.416813 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401de8b-9d29-4e66-8a69-a27b5072dff0 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:41.480712 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b4e5f-6a47-479c-869c-f2c21ed2e45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:41.481608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b4e5f-6a47-479c-869c-f2c21ed2e45a admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.481933 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b4e5f-6a47-479c-869c-f2c21ed2e45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.482297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b4e5f-6a47-479c-869c-f2c21ed2e45a admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.483283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b4e5f-6a47-479c-869c-f2c21ed2e45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:41.486172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b4e5f-6a47-479c-869c-f2c21ed2e45a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:41.509557 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1306/2581] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:41 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:18:41.513849 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12427b-75db-491e-9047-0672d7e21678 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:41.524554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12427b-75db-491e-9047-0672d7e21678 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:41.530432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669d14f-aeb6-4940-8ec3-30b3561bd527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:41.531096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669d14f-aeb6-4940-8ec3-30b3561bd527 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.531380 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669d14f-aeb6-4940-8ec3-30b3561bd527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.531651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669d14f-aeb6-4940-8ec3-30b3561bd527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.532385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669d14f-aeb6-4940-8ec3-30b3561bd527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:41.534004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669d14f-aeb6-4940-8ec3-30b3561bd527 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:41.544894 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1276/2582] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:41.548808 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b4e5f-6a47-479c-869c-f2c21ed2e45a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:41.558895 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b4e5f-6a47-479c-869c-f2c21ed2e45a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:41.563533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6e7fb-6071-43fb-819a-04b87493b47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:41.564188 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6e7fb-6071-43fb-819a-04b87493b47b admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.564468 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6e7fb-6071-43fb-819a-04b87493b47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.564735 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6e7fb-6071-43fb-819a-04b87493b47b admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.565455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6e7fb-6071-43fb-819a-04b87493b47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:41.567087 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6e7fb-6071-43fb-819a-04b87493b47b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:41.577222 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1307/2583] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:41.583194 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669d14f-aeb6-4940-8ec3-30b3561bd527 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:41.594340 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669d14f-aeb6-4940-8ec3-30b3561bd527 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:41.600252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06e2a9-9aa0-4a7b-a14e-67a644317f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:41.601141 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e2a9-9aa0-4a7b-a14e-67a644317f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.601422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e2a9-9aa0-4a7b-a14e-67a644317f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.601714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e2a9-9aa0-4a7b-a14e-67a644317f3c admin admin] PATH_INFO: `/v3/projects/c778a164ebf94ccfb1969025c1d3a5c0/users/95ffbea8de4447a58b216ed98caa76a1/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.617096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06e2a9-9aa0-4a7b-a14e-67a644317f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c778a164ebf94ccfb1969025c1d3a5c0, user_id=95ffbea8de4447a58b216ed98caa76a1, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:41.619725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06e2a9-9aa0-4a7b-a14e-67a644317f3c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:41.634755 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1277/2584] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:41 2022] PUT /identity/v3/projects/c778a164ebf94ccfb1969025c1d3a5c0/users/95ffbea8de4447a58b216ed98caa76a1/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:41.645233 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77272762-9acc-41fd-88c6-dd620a58086b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.645551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77272762-9acc-41fd-88c6-dd620a58086b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.645823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77272762-9acc-41fd-88c6-dd620a58086b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.672630 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-77272762-9acc-41fd-88c6-dd620a58086b None None] MFA Rules not processed for user `95ffbea8de4447a58b216ed98caa76a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:41.673838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-77272762-9acc-41fd-88c6-dd620a58086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:41.741111 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1308/2585] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:41.747702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06e2a9-9aa0-4a7b-a14e-67a644317f3c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:41.814811 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06e2a9-9aa0-4a7b-a14e-67a644317f3c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:41.820233 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98d4ed-1a11-4abc-8d6e-31b9f6077f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:41.820893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d4ed-1a11-4abc-8d6e-31b9f6077f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.821183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d4ed-1a11-4abc-8d6e-31b9f6077f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.821460 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d4ed-1a11-4abc-8d6e-31b9f6077f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.822146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98d4ed-1a11-4abc-8d6e-31b9f6077f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:41.824084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98d4ed-1a11-4abc-8d6e-31b9f6077f88 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:41.860383 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1278/2586] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:41 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:41.865183 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77272762-9acc-41fd-88c6-dd620a58086b None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:41.878592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77272762-9acc-41fd-88c6-dd620a58086b None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:41.906440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:41.937580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b9c7d-b32c-493f-9a9a-48ed7bfc41a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:41.938304 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b9c7d-b32c-493f-9a9a-48ed7bfc41a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.938591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b9c7d-b32c-493f-9a9a-48ed7bfc41a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.938871 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b9c7d-b32c-493f-9a9a-48ed7bfc41a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.939562 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b9c7d-b32c-493f-9a9a-48ed7bfc41a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:41.941695 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b9c7d-b32c-493f-9a9a-48ed7bfc41a2 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:41.961846 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1309/2587] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:41 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:41.966488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98d4ed-1a11-4abc-8d6e-31b9f6077f88 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:41.977226 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98d4ed-1a11-4abc-8d6e-31b9f6077f88 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:41.983222 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99679a45-76f5-49cf-8469-1ce032889270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:41.983937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99679a45-76f5-49cf-8469-1ce032889270 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:41.984222 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99679a45-76f5-49cf-8469-1ce032889270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:41.984496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99679a45-76f5-49cf-8469-1ce032889270 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:41.985224 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99679a45-76f5-49cf-8469-1ce032889270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:41.986903 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99679a45-76f5-49cf-8469-1ce032889270 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:41.997747 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1279/2588] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:42.001930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b9c7d-b32c-493f-9a9a-48ed7bfc41a2 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:42.012665 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b9c7d-b32c-493f-9a9a-48ed7bfc41a2 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:42.018268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bf701-50a3-4ca6-b909-7c78951211ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:42.019181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bf701-50a3-4ca6-b909-7c78951211ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:42.019463 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bf701-50a3-4ca6-b909-7c78951211ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:42.019739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bf701-50a3-4ca6-b909-7c78951211ae admin admin] PATH_INFO: `/v3/projects/f8fb5332288a425c842102522a6fbe6e/users/11ddee1253a644d782287c6bd495cd09/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:42.035223 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7bf701-50a3-4ca6-b909-7c78951211ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8fb5332288a425c842102522a6fbe6e, user_id=11ddee1253a644d782287c6bd495cd09, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:42.037859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7bf701-50a3-4ca6-b909-7c78951211ae admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:42.052259 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1310/2589] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:42 2022] PUT /identity/v3/projects/f8fb5332288a425c842102522a6fbe6e/users/11ddee1253a644d782287c6bd495cd09/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:42.056278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99679a45-76f5-49cf-8469-1ce032889270 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:42.066662 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99679a45-76f5-49cf-8469-1ce032889270 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:42.126871 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a512920-7142-4722-81ac-8be7a76dc99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:42.127556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512920-7142-4722-81ac-8be7a76dc99f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:42.127845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512920-7142-4722-81ac-8be7a76dc99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:42.128123 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512920-7142-4722-81ac-8be7a76dc99f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:42.128907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a512920-7142-4722-81ac-8be7a76dc99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:42.130511 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a512920-7142-4722-81ac-8be7a76dc99f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:42.131740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:42.145834 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1280/2590] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:42.147045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bf701-50a3-4ca6-b909-7c78951211ae admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:42.157381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bf701-50a3-4ca6-b909-7c78951211ae admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:42.162132 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8827f-c73c-4f82-a3f6-e2bbcae64ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:42.163073 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8827f-c73c-4f82-a3f6-e2bbcae64ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:42.163358 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8827f-c73c-4f82-a3f6-e2bbcae64ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:42.163633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8827f-c73c-4f82-a3f6-e2bbcae64ef3 admin admin] PATH_INFO: `/v3/projects/f8fb5332288a425c842102522a6fbe6e/users/11ddee1253a644d782287c6bd495cd09/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:42.165646 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8827f-c73c-4f82-a3f6-e2bbcae64ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8fb5332288a425c842102522a6fbe6e, user_id=11ddee1253a644d782287c6bd495cd09, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:42.168388 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8827f-c73c-4f82-a3f6-e2bbcae64ef3 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:42.178762 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1311/2591] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:42 2022] PUT /identity/v3/projects/f8fb5332288a425c842102522a6fbe6e/users/11ddee1253a644d782287c6bd495cd09/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:18:42.190027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e9088-153f-4521-bbe3-693fb1bf27f2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:42.190354 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e9088-153f-4521-bbe3-693fb1bf27f2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:42.190636 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e9088-153f-4521-bbe3-693fb1bf27f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:42.219513 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c66e9088-153f-4521-bbe3-693fb1bf27f2 None None] MFA Rules not processed for user `11ddee1253a644d782287c6bd495cd09`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:42.220757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c66e9088-153f-4521-bbe3-693fb1bf27f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:42.291874 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1281/2592] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:18:42.303828 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8827f-c73c-4f82-a3f6-e2bbcae64ef3 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:42.306051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3ec8827f-c73c-4f82-a3f6-e2bbcae64ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:42.370201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8827f-c73c-4f82-a3f6-e2bbcae64ef3 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:42.374699 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c308f6-86d2-436e-a008-19595fc0475b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:42.375354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c308f6-86d2-436e-a008-19595fc0475b service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:42.375606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c308f6-86d2-436e-a008-19595fc0475b service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:42.375942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c308f6-86d2-436e-a008-19595fc0475b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:42.385549 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c308f6-86d2-436e-a008-19595fc0475b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:42.386961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c308f6-86d2-436e-a008-19595fc0475b service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:42.396475 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1312/2593] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:43.804576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e9088-153f-4521-bbe3-693fb1bf27f2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:43.870088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e9088-153f-4521-bbe3-693fb1bf27f2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:43.875878 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:43.876774 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:43.877059 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:43.877333 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] PATH_INFO: `/v3/users/95ffbea8de4447a58b216ed98caa76a1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:43.878730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ffbea8de4447a58b216ed98caa76a1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:43.880715 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:43.930822 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ffbea8de4447a58b216ed98caa76a1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:43.931418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ffbea8de4447a58b216ed98caa76a1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:43.932044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ffbea8de4447a58b216ed98caa76a1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:43.939118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ffbea8de4447a58b216ed98caa76a1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:43.950747 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ffbea8de4447a58b216ed98caa76a1'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:43.961468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:43.966546 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1282/2594] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:43 2022] DELETE /identity/v3/users/95ffbea8de4447a58b216ed98caa76a1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:43.971521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17297c-18fe-40dd-af86-e1ad3e395bc1 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:43.972044 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17297c-18fe-40dd-af86-e1ad3e395bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:43.972172 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17297c-18fe-40dd-af86-e1ad3e395bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:43.989728 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-ca17297c-18fe-40dd-af86-e1ad3e395bc1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:43.990974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ca17297c-18fe-40dd-af86-e1ad3e395bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:44.058256 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1313/2595] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:44.068234 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:44.069787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:44.130617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9f577-89b1-46e8-91cf-7c5fca58b87e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:44.135320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b83da-7fbf-42dd-997d-d3a8106e0887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:44.135973 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b83da-7fbf-42dd-997d-d3a8106e0887 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:44.136255 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b83da-7fbf-42dd-997d-d3a8106e0887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:44.136529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b83da-7fbf-42dd-997d-d3a8106e0887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:44.146261 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b83da-7fbf-42dd-997d-d3a8106e0887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:44.147733 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b83da-7fbf-42dd-997d-d3a8106e0887 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:44.157748 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1283/2596] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:44.436252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17297c-18fe-40dd-af86-e1ad3e395bc1 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:44.437863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-ca17297c-18fe-40dd-af86-e1ad3e395bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:44.450739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17297c-18fe-40dd-af86-e1ad3e395bc1 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:44.455586 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:44.456467 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:44.456749 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:44.457018 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] PATH_INFO: `/v3/projects/c778a164ebf94ccfb1969025c1d3a5c0` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:44.458357 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c778a164ebf94ccfb1969025c1d3a5c0)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:44.460256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:44.482809 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:44.508872 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c778a164ebf94ccfb1969025c1d3a5c0', 'request_id': 'req-526d7460-0168-49a3-a5d0-d072c5fb154a'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:44.514304 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c778a164ebf94ccfb1969025c1d3a5c0', 'request_id': 'req-526d7460-0168-49a3-a5d0-d072c5fb154a'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:44.520192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c778a164ebf94ccfb1969025c1d3a5c0', 'request_id': 'req-526d7460-0168-49a3-a5d0-d072c5fb154a'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:44.533657 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] The token cache is being invalidate because project c778a164ebf94ccfb1969025c1d3a5c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:44.533942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:44.534575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:44.536100 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1314/2597] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:44 2022] DELETE /identity/v3/projects/c778a164ebf94ccfb1969025c1d3a5c0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:18:44.540279 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b83da-7fbf-42dd-997d-d3a8106e0887 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:44.549894 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b83da-7fbf-42dd-997d-d3a8106e0887 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:44.614526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d0641-50c8-4990-b10b-ba5e35c460fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:44.615402 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d0641-50c8-4990-b10b-ba5e35c460fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:44.615687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d0641-50c8-4990-b10b-ba5e35c460fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:44.615964 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d0641-50c8-4990-b10b-ba5e35c460fd admin admin] PATH_INFO: `/v3/users/11ddee1253a644d782287c6bd495cd09` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:44.617175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d0641-50c8-4990-b10b-ba5e35c460fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=11ddee1253a644d782287c6bd495cd09)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:44.619189 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d0641-50c8-4990-b10b-ba5e35c460fd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:44.666515 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-312d0641-50c8-4990-b10b-ba5e35c460fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11ddee1253a644d782287c6bd495cd09'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:44.667159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-312d0641-50c8-4990-b10b-ba5e35c460fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11ddee1253a644d782287c6bd495cd09'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:44.667761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-312d0641-50c8-4990-b10b-ba5e35c460fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11ddee1253a644d782287c6bd495cd09'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:44.674461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-312d0641-50c8-4990-b10b-ba5e35c460fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11ddee1253a644d782287c6bd495cd09'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:44.685694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-312d0641-50c8-4990-b10b-ba5e35c460fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11ddee1253a644d782287c6bd495cd09'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:44.699803 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1284/2598] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:44 2022] DELETE /identity/v3/users/11ddee1253a644d782287c6bd495cd09 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:44.984325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:44.985968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:45.000555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d7460-0168-49a3-a5d0-d072c5fb154a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:45.057087 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:45.057939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.058279 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.058557 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] PATH_INFO: `/v3/projects/f8fb5332288a425c842102522a6fbe6e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.059796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8fb5332288a425c842102522a6fbe6e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:45.061730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:45.109224 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8fb5332288a425c842102522a6fbe6e', 'request_id': 'req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:45.114455 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8fb5332288a425c842102522a6fbe6e', 'request_id': 'req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:45.120471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8fb5332288a425c842102522a6fbe6e', 'request_id': 'req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:45.135553 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] The token cache is being invalidate because project f8fb5332288a425c842102522a6fbe6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:45.135842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:45.136487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4b3070a6-3dae-49e5-b48d-6b8ad3d37db7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:45.138046 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1315/2599] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:44 2022] DELETE /identity/v3/projects/f8fb5332288a425c842102522a6fbe6e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:45.143664 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c2de4-8ba2-4b74-b234-baf68d42c192 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.144157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c2de4-8ba2-4b74-b234-baf68d42c192 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.144252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c2de4-8ba2-4b74-b234-baf68d42c192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.163109 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-901c2de4-8ba2-4b74-b234-baf68d42c192 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:45.166445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-901c2de4-8ba2-4b74-b234-baf68d42c192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:45.185009 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:45.230962 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1285/2600] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:45.239046 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279e632-a0fc-4a30-a9a9-b035f0cc8076 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.239340 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279e632-a0fc-4a30-a9a9-b035f0cc8076 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.239628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279e632-a0fc-4a30-a9a9-b035f0cc8076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.259502 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-2279e632-a0fc-4a30-a9a9-b035f0cc8076 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:45.260756 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2279e632-a0fc-4a30-a9a9-b035f0cc8076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:45.270254 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1316/2601] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:45.274612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c2de4-8ba2-4b74-b234-baf68d42c192 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:45.284649 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c2de4-8ba2-4b74-b234-baf68d42c192 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:45.290594 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b749da2-c173-46a9-a269-2b22124f7d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:45.291275 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b749da2-c173-46a9-a269-2b22124f7d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.291556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b749da2-c173-46a9-a269-2b22124f7d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.291831 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b749da2-c173-46a9-a269-2b22124f7d98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.292556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b749da2-c173-46a9-a269-2b22124f7d98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:18:45.292858 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b749da2-c173-46a9-a269-2b22124f7d98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:45.294652 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b749da2-c173-46a9-a269-2b22124f7d98 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:45.306480 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1286/2602] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:18:45.310888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279e632-a0fc-4a30-a9a9-b035f0cc8076 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:45.320466 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279e632-a0fc-4a30-a9a9-b035f0cc8076 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:45.325974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ce71e-ab2d-4051-ab0e-5a025378b3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:45.326667 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ce71e-ab2d-4051-ab0e-5a025378b3ad admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.326960 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ce71e-ab2d-4051-ab0e-5a025378b3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.327208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ce71e-ab2d-4051-ab0e-5a025378b3ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.327913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ce71e-ab2d-4051-ab0e-5a025378b3ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:45.329866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ce71e-ab2d-4051-ab0e-5a025378b3ad admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:45.358114 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1317/2603] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:45 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:45.362570 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b749da2-c173-46a9-a269-2b22124f7d98 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:45.372774 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b749da2-c173-46a9-a269-2b22124f7d98 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:45.431293 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ac3d4-4d86-4413-a711-2b5b93aca27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:45.431970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ac3d4-4d86-4413-a711-2b5b93aca27a admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.432288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ac3d4-4d86-4413-a711-2b5b93aca27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.432575 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ac3d4-4d86-4413-a711-2b5b93aca27a admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.433280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ac3d4-4d86-4413-a711-2b5b93aca27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:45.435272 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ac3d4-4d86-4413-a711-2b5b93aca27a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:45.456614 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1287/2604] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:45 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:45.461204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ce71e-ab2d-4051-ab0e-5a025378b3ad admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:45.472316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ce71e-ab2d-4051-ab0e-5a025378b3ad admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:45.477996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582a329-fb81-474c-b65d-e22b1aa91eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:45.478716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582a329-fb81-474c-b65d-e22b1aa91eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.478998 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582a329-fb81-474c-b65d-e22b1aa91eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.479270 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582a329-fb81-474c-b65d-e22b1aa91eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.479996 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582a329-fb81-474c-b65d-e22b1aa91eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:45.481643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582a329-fb81-474c-b65d-e22b1aa91eb8 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:45.483762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:45.494047 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1318/2605] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:45.498431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ac3d4-4d86-4413-a711-2b5b93aca27a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:45.508495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ac3d4-4d86-4413-a711-2b5b93aca27a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:45.513318 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8400c-12e4-4add-8783-f1c6feba40a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:45.513974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8400c-12e4-4add-8783-f1c6feba40a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.514285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8400c-12e4-4add-8783-f1c6feba40a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.514561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8400c-12e4-4add-8783-f1c6feba40a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.515324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8400c-12e4-4add-8783-f1c6feba40a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:45.516918 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8400c-12e4-4add-8783-f1c6feba40a2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:45.527112 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1288/2606] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:45.532477 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582a329-fb81-474c-b65d-e22b1aa91eb8 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:45.547044 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582a329-fb81-474c-b65d-e22b1aa91eb8 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:45.555672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e792756-82a6-4c7e-9903-a5fcec383657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:45.556588 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e792756-82a6-4c7e-9903-a5fcec383657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.556874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e792756-82a6-4c7e-9903-a5fcec383657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.557149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e792756-82a6-4c7e-9903-a5fcec383657 admin admin] PATH_INFO: `/v3/projects/5bc114f8fce0412b8fc8acf01b1b2a0f/users/2b3d5660cece4348a65a62abc31c5bce/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.573100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e792756-82a6-4c7e-9903-a5fcec383657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc114f8fce0412b8fc8acf01b1b2a0f, user_id=2b3d5660cece4348a65a62abc31c5bce, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:45.575801 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e792756-82a6-4c7e-9903-a5fcec383657 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:45.592979 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1319/2607] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:45 2022] PUT /identity/v3/projects/5bc114f8fce0412b8fc8acf01b1b2a0f/users/2b3d5660cece4348a65a62abc31c5bce/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:45.608227 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799c7d7-dea5-4f53-81c7-64ac366de171 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.608507 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799c7d7-dea5-4f53-81c7-64ac366de171 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.608790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799c7d7-dea5-4f53-81c7-64ac366de171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.638722 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1799c7d7-dea5-4f53-81c7-64ac366de171 None None] MFA Rules not processed for user `2b3d5660cece4348a65a62abc31c5bce`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:45.639937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1799c7d7-dea5-4f53-81c7-64ac366de171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:45.705961 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1289/2608] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:45.711156 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e792756-82a6-4c7e-9903-a5fcec383657 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:45.783012 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e792756-82a6-4c7e-9903-a5fcec383657 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:45.788196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600a080-549e-449b-a7b5-57d6500c04fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:45.788858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600a080-549e-449b-a7b5-57d6500c04fc admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.789136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600a080-549e-449b-a7b5-57d6500c04fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.789410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600a080-549e-449b-a7b5-57d6500c04fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.790082 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600a080-549e-449b-a7b5-57d6500c04fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:45.792021 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600a080-549e-449b-a7b5-57d6500c04fc admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:45.820809 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1320/2609] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:45 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:45.825098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799c7d7-dea5-4f53-81c7-64ac366de171 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:45.836383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799c7d7-dea5-4f53-81c7-64ac366de171 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:45.849468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:45.894440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e15d20-6bf5-4496-9c57-fa850f721e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:45.895131 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e15d20-6bf5-4496-9c57-fa850f721e47 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.895420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e15d20-6bf5-4496-9c57-fa850f721e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.895707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e15d20-6bf5-4496-9c57-fa850f721e47 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.896423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e15d20-6bf5-4496-9c57-fa850f721e47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:45.898456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e15d20-6bf5-4496-9c57-fa850f721e47 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:45.920854 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1290/2610] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:45 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:45.925151 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600a080-549e-449b-a7b5-57d6500c04fc admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:45.935784 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600a080-549e-449b-a7b5-57d6500c04fc admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:45.940781 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc21c04-8372-4bd4-8681-6ba69f7c0995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:45.941453 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc21c04-8372-4bd4-8681-6ba69f7c0995 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.941740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc21c04-8372-4bd4-8681-6ba69f7c0995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.942019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc21c04-8372-4bd4-8681-6ba69f7c0995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.942791 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc21c04-8372-4bd4-8681-6ba69f7c0995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:45.944432 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc21c04-8372-4bd4-8681-6ba69f7c0995 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:45.955431 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1321/2611] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:45.959392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e15d20-6bf5-4496-9c57-fa850f721e47 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:45.969998 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e15d20-6bf5-4496-9c57-fa850f721e47 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:45.975690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b772c04-b95c-433a-b472-e87fc321c513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:45.976407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b772c04-b95c-433a-b472-e87fc321c513 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:45.976694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b772c04-b95c-433a-b472-e87fc321c513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:45.976972 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b772c04-b95c-433a-b472-e87fc321c513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:45.977704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b772c04-b95c-433a-b472-e87fc321c513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:45.979359 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b772c04-b95c-433a-b472-e87fc321c513 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:45.989784 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1291/2612] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:45.994080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc21c04-8372-4bd4-8681-6ba69f7c0995 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:46.004358 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc21c04-8372-4bd4-8681-6ba69f7c0995 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:46.010200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c19738-1564-4a30-aa0d-eed43ec1ddcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:46.011138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c19738-1564-4a30-aa0d-eed43ec1ddcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:46.011409 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c19738-1564-4a30-aa0d-eed43ec1ddcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:46.011710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c19738-1564-4a30-aa0d-eed43ec1ddcd admin admin] PATH_INFO: `/v3/projects/abb313975c81467a8c767d70e1651f10/users/d4217ca9e01d418eb946349cb562ba79/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:46.028379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c19738-1564-4a30-aa0d-eed43ec1ddcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb313975c81467a8c767d70e1651f10, user_id=d4217ca9e01d418eb946349cb562ba79, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:46.031158 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c19738-1564-4a30-aa0d-eed43ec1ddcd admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:46.047141 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1322/2613] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:45 2022] PUT /identity/v3/projects/abb313975c81467a8c767d70e1651f10/users/d4217ca9e01d418eb946349cb562ba79/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:46.115303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc516df3-add8-4d07-acbb-7eb52d4de7a4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:46.115608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc516df3-add8-4d07-acbb-7eb52d4de7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:46.115957 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc516df3-add8-4d07-acbb-7eb52d4de7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:46.144945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fc516df3-add8-4d07-acbb-7eb52d4de7a4 None None] MFA Rules not processed for user `d4217ca9e01d418eb946349cb562ba79`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:46.146142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fc516df3-add8-4d07-acbb-7eb52d4de7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:46.218745 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1292/2614] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:46.223524 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c19738-1564-4a30-aa0d-eed43ec1ddcd admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:46.288648 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c19738-1564-4a30-aa0d-eed43ec1ddcd admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:46.293850 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f0111-021e-44a0-b212-59876b2e8941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:46.294545 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f0111-021e-44a0-b212-59876b2e8941 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:46.294830 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f0111-021e-44a0-b212-59876b2e8941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:46.295106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f0111-021e-44a0-b212-59876b2e8941 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:46.295795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f0111-021e-44a0-b212-59876b2e8941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:46.297707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f0111-021e-44a0-b212-59876b2e8941 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:46.325433 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1323/2615] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:46 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:46.331788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc516df3-add8-4d07-acbb-7eb52d4de7a4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:46.343202 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc516df3-add8-4d07-acbb-7eb52d4de7a4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:46.398196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ba52a-bec3-466f-98ee-d19a94c674b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:46.398903 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ba52a-bec3-466f-98ee-d19a94c674b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:46.399177 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ba52a-bec3-466f-98ee-d19a94c674b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:46.399446 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ba52a-bec3-466f-98ee-d19a94c674b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:46.400194 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ba52a-bec3-466f-98ee-d19a94c674b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:46.402124 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ba52a-bec3-466f-98ee-d19a94c674b9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:46.423947 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1293/2616] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:46 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:46.428120 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f0111-021e-44a0-b212-59876b2e8941 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:46.438891 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f0111-021e-44a0-b212-59876b2e8941 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:46.443563 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7feba7-ba09-4a51-82da-9ee3a37cc5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:46.444245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7feba7-ba09-4a51-82da-9ee3a37cc5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:46.444536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7feba7-ba09-4a51-82da-9ee3a37cc5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:46.444800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7feba7-ba09-4a51-82da-9ee3a37cc5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:46.445535 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7feba7-ba09-4a51-82da-9ee3a37cc5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:46.447213 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7feba7-ba09-4a51-82da-9ee3a37cc5f1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:46.449383 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:46.458808 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1324/2617] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:46.462878 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ba52a-bec3-466f-98ee-d19a94c674b9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:46.473461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ba52a-bec3-466f-98ee-d19a94c674b9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:46.478591 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e057bf-19c1-4501-a9cc-29682a35aa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:46.479497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e057bf-19c1-4501-a9cc-29682a35aa81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:46.479771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e057bf-19c1-4501-a9cc-29682a35aa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:46.480042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e057bf-19c1-4501-a9cc-29682a35aa81 admin admin] PATH_INFO: `/v3/projects/e2471507255d41408061b6e3782ae0e4/users/292bcac6a5e74cd49e95cda01083dbe3/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:46.492651 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:46.495906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e057bf-19c1-4501-a9cc-29682a35aa81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2471507255d41408061b6e3782ae0e4, user_id=292bcac6a5e74cd49e95cda01083dbe3, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:46.498717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e057bf-19c1-4501-a9cc-29682a35aa81 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:46.513791 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1294/2618] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:46 2022] PUT /identity/v3/projects/e2471507255d41408061b6e3782ae0e4/users/292bcac6a5e74cd49e95cda01083dbe3/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:46.517917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7feba7-ba09-4a51-82da-9ee3a37cc5f1 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:46.528649 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7feba7-ba09-4a51-82da-9ee3a37cc5f1 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:46.588200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6abec-3eda-47c0-b339-93e5ccadfe48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:46.589241 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6abec-3eda-47c0-b339-93e5ccadfe48 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:46.589524 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6abec-3eda-47c0-b339-93e5ccadfe48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:46.589799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6abec-3eda-47c0-b339-93e5ccadfe48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:46.590556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6abec-3eda-47c0-b339-93e5ccadfe48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:46.592169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6abec-3eda-47c0-b339-93e5ccadfe48 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:46.601921 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1325/2619] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:46.606348 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e057bf-19c1-4501-a9cc-29682a35aa81 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:46.617865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e057bf-19c1-4501-a9cc-29682a35aa81 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:46.624152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793c1dd-adeb-4985-bf71-dabaab8e6a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:46.625049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793c1dd-adeb-4985-bf71-dabaab8e6a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:46.625334 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793c1dd-adeb-4985-bf71-dabaab8e6a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:46.625612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793c1dd-adeb-4985-bf71-dabaab8e6a75 admin admin] PATH_INFO: `/v3/projects/e2471507255d41408061b6e3782ae0e4/users/292bcac6a5e74cd49e95cda01083dbe3/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:46.627837 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793c1dd-adeb-4985-bf71-dabaab8e6a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2471507255d41408061b6e3782ae0e4, user_id=292bcac6a5e74cd49e95cda01083dbe3, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:46.630526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793c1dd-adeb-4985-bf71-dabaab8e6a75 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:46.641683 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1295/2620] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:46 2022] PUT /identity/v3/projects/e2471507255d41408061b6e3782ae0e4/users/292bcac6a5e74cd49e95cda01083dbe3/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:46.652103 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f0d6-1bdf-41ba-a75c-986985e0c253 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:46.652294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f0d6-1bdf-41ba-a75c-986985e0c253 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:46.652595 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f0d6-1bdf-41ba-a75c-986985e0c253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:46.680348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e231f0d6-1bdf-41ba-a75c-986985e0c253 None None] MFA Rules not processed for user `292bcac6a5e74cd49e95cda01083dbe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:46.681546 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e231f0d6-1bdf-41ba-a75c-986985e0c253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:46.754259 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1326/2621] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:46.767892 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793c1dd-adeb-4985-bf71-dabaab8e6a75 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:46.769805 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-6793c1dd-adeb-4985-bf71-dabaab8e6a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:46.832086 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793c1dd-adeb-4985-bf71-dabaab8e6a75 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:46.836248 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cab373-56ee-4829-8ae2-e6ff79a971af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:46.836909 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cab373-56ee-4829-8ae2-e6ff79a971af service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:46.837195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cab373-56ee-4829-8ae2-e6ff79a971af service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:46.837479 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cab373-56ee-4829-8ae2-e6ff79a971af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:46.847102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cab373-56ee-4829-8ae2-e6ff79a971af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:46.848540 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cab373-56ee-4829-8ae2-e6ff79a971af service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:46.858334 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1296/2622] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:47.372023 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231f0d6-1bdf-41ba-a75c-986985e0c253 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:47.373681 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e231f0d6-1bdf-41ba-a75c-986985e0c253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:47.387872 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231f0d6-1bdf-41ba-a75c-986985e0c253 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:47.392739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d86860-0322-4127-91a1-e65fe9399bc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:47.393375 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d86860-0322-4127-91a1-e65fe9399bc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:47.393654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d86860-0322-4127-91a1-e65fe9399bc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:47.393929 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d86860-0322-4127-91a1-e65fe9399bc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:47.452347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d86860-0322-4127-91a1-e65fe9399bc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:47.453724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d86860-0322-4127-91a1-e65fe9399bc2 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:47.462825 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1327/2623] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:51.514814 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cab373-56ee-4829-8ae2-e6ff79a971af service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:51.524019 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cab373-56ee-4829-8ae2-e6ff79a971af service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:51.528683 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba678d4e-125f-429c-bd95-6cb6421d1941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:51.529361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678d4e-125f-429c-bd95-6cb6421d1941 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:51.529649 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678d4e-125f-429c-bd95-6cb6421d1941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:51.529973 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678d4e-125f-429c-bd95-6cb6421d1941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:51.585809 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba678d4e-125f-429c-bd95-6cb6421d1941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:51.587221 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba678d4e-125f-429c-bd95-6cb6421d1941 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:51.597107 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1297/2624] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:53.732370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d86860-0322-4127-91a1-e65fe9399bc2 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:53.751902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:53.795656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d86860-0322-4127-91a1-e65fe9399bc2 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:53.800605 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:53.801549 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:53.801831 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:53.802105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] PATH_INFO: `/v3/users/2b3d5660cece4348a65a62abc31c5bce` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:53.803342 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b3d5660cece4348a65a62abc31c5bce)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:53.805269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:53.853454 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b3d5660cece4348a65a62abc31c5bce'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:53.854058 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b3d5660cece4348a65a62abc31c5bce'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:53.866417 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b3d5660cece4348a65a62abc31c5bce'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:53.873384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b3d5660cece4348a65a62abc31c5bce'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:53.874032 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b3d5660cece4348a65a62abc31c5bce'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:53.889670 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1328/2625] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:53 2022] DELETE /identity/v3/users/2b3d5660cece4348a65a62abc31c5bce => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:53.894381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af387b57-7a99-4ee0-9c8a-80e7dbc87eb8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:53.894873 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af387b57-7a99-4ee0-9c8a-80e7dbc87eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:53.895007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af387b57-7a99-4ee0-9c8a-80e7dbc87eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:53.913018 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-af387b57-7a99-4ee0-9c8a-80e7dbc87eb8 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:53.914262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-af387b57-7a99-4ee0-9c8a-80e7dbc87eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:53.974677 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1298/2626] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:53.984887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:53.986388 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:54.046755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8540d-3892-42a6-a0d8-9e32de24f831 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:54.051185 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6ece8-7f84-48dc-822d-3f657bf07a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:54.051848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6ece8-7f84-48dc-822d-3f657bf07a88 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:54.052130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6ece8-7f84-48dc-822d-3f657bf07a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:54.052403 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6ece8-7f84-48dc-822d-3f657bf07a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:54.062777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6ece8-7f84-48dc-822d-3f657bf07a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:54.064241 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6ece8-7f84-48dc-822d-3f657bf07a88 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:54.074133 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1329/2627] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:18:54.351224 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af387b57-7a99-4ee0-9c8a-80e7dbc87eb8 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:54.352970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-af387b57-7a99-4ee0-9c8a-80e7dbc87eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:54.367394 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af387b57-7a99-4ee0-9c8a-80e7dbc87eb8 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:54.373391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:54.374515 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:54.374700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:54.375086 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] PATH_INFO: `/v3/projects/5bc114f8fce0412b8fc8acf01b1b2a0f` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:54.376530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc114f8fce0412b8fc8acf01b1b2a0f)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:54.378527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:54.396075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:54.430243 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc114f8fce0412b8fc8acf01b1b2a0f', 'request_id': 'req-97abff06-c68d-4f14-b84e-4c98a4e8ee88'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:54.443316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc114f8fce0412b8fc8acf01b1b2a0f', 'request_id': 'req-97abff06-c68d-4f14-b84e-4c98a4e8ee88'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:54.449445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc114f8fce0412b8fc8acf01b1b2a0f', 'request_id': 'req-97abff06-c68d-4f14-b84e-4c98a4e8ee88'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:54.460866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] The token cache is being invalidate because project 5bc114f8fce0412b8fc8acf01b1b2a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:54.461179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:54.461816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:54.463780 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1299/2628] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:54 2022] DELETE /identity/v3/projects/5bc114f8fce0412b8fc8acf01b1b2a0f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:54.468177 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6ece8-7f84-48dc-822d-3f657bf07a88 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:54.478242 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6ece8-7f84-48dc-822d-3f657bf07a88 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:54.531300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:54.552078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:54.553006 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:54.553294 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:54.553573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] PATH_INFO: `/v3/users/d4217ca9e01d418eb946349cb562ba79` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:54.554798 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4217ca9e01d418eb946349cb562ba79)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:54.556807 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:54.603740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4217ca9e01d418eb946349cb562ba79'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:54.604407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4217ca9e01d418eb946349cb562ba79'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:54.618430 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4217ca9e01d418eb946349cb562ba79'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:54.625572 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4217ca9e01d418eb946349cb562ba79'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:54.626290 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4217ca9e01d418eb946349cb562ba79'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:54.641178 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1330/2629] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:54 2022] DELETE /identity/v3/users/d4217ca9e01d418eb946349cb562ba79 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:54.921206 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:54.922839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:54.936483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abff06-c68d-4f14-b84e-4c98a4e8ee88 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:54.994787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:54.995670 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:54.995965 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:54.996225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] PATH_INFO: `/v3/projects/abb313975c81467a8c767d70e1651f10` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:54.997449 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb313975c81467a8c767d70e1651f10)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:54.999432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:55.046242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb313975c81467a8c767d70e1651f10', 'request_id': 'req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.054921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb313975c81467a8c767d70e1651f10', 'request_id': 'req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.060324 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb313975c81467a8c767d70e1651f10', 'request_id': 'req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.071499 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] The token cache is being invalidate because project abb313975c81467a8c767d70e1651f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:55.071792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.072578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.074360 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1300/2630] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:54 2022] DELETE /identity/v3/projects/abb313975c81467a8c767d70e1651f10 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:55.079089 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:55.081245 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:55.097576 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b62c0-c26d-41ce-a40a-9ccbfde93916 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:55.172888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3f1f2-d111-4ed9-8aa8-43e1e7c2df48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:55.173941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3f1f2-d111-4ed9-8aa8-43e1e7c2df48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:55.174256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3f1f2-d111-4ed9-8aa8-43e1e7c2df48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:55.174579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3f1f2-d111-4ed9-8aa8-43e1e7c2df48 admin admin] PATH_INFO: `/v3/users/292bcac6a5e74cd49e95cda01083dbe3` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:55.176372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3f1f2-d111-4ed9-8aa8-43e1e7c2df48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=292bcac6a5e74cd49e95cda01083dbe3)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:55.179262 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3f1f2-d111-4ed9-8aa8-43e1e7c2df48 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:55.230525 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-8ed3f1f2-d111-4ed9-8aa8-43e1e7c2df48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '292bcac6a5e74cd49e95cda01083dbe3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.231239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-8ed3f1f2-d111-4ed9-8aa8-43e1e7c2df48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '292bcac6a5e74cd49e95cda01083dbe3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.241227 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:55.243404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-8ed3f1f2-d111-4ed9-8aa8-43e1e7c2df48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '292bcac6a5e74cd49e95cda01083dbe3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.251558 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-8ed3f1f2-d111-4ed9-8aa8-43e1e7c2df48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '292bcac6a5e74cd49e95cda01083dbe3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.252683 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-8ed3f1f2-d111-4ed9-8aa8-43e1e7c2df48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '292bcac6a5e74cd49e95cda01083dbe3'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.265298 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1331/2631] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:18:55 2022] DELETE /identity/v3/users/292bcac6a5e74cd49e95cda01083dbe3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:55.551567 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:55.553102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:55.566526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cfe04-7cca-4b32-b0b4-a1e012f24211 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:55.597723 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:55.626091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:55.627014 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:55.627278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:55.627578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] PATH_INFO: `/v3/projects/e2471507255d41408061b6e3782ae0e4` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:55.628783 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2471507255d41408061b6e3782ae0e4)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:55.630731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:55.681789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2471507255d41408061b6e3782ae0e4', 'request_id': 'req-736c17f0-12e0-4b66-888d-a3b915fe9dfd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.691610 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2471507255d41408061b6e3782ae0e4', 'request_id': 'req-736c17f0-12e0-4b66-888d-a3b915fe9dfd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.697383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2471507255d41408061b6e3782ae0e4', 'request_id': 'req-736c17f0-12e0-4b66-888d-a3b915fe9dfd'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.708232 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] The token cache is being invalidate because project e2471507255d41408061b6e3782ae0e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:18:55.708519 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.709138 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-736c17f0-12e0-4b66-888d-a3b915fe9dfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:18:55.710760 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1301/2632] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:18:55 2022] DELETE /identity/v3/projects/e2471507255d41408061b6e3782ae0e4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:55.717242 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e467a2-cb41-4bfd-acee-5dbc33a0b1ed None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:55.717551 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e467a2-cb41-4bfd-acee-5dbc33a0b1ed None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:55.717825 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e467a2-cb41-4bfd-acee-5dbc33a0b1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:55.736385 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a5e467a2-cb41-4bfd-acee-5dbc33a0b1ed None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:55.737589 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a5e467a2-cb41-4bfd-acee-5dbc33a0b1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:55.798874 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1332/2633] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:18:55.807604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9ac68-a647-4c89-8b4c-ba87af80d573 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:55.807886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9ac68-a647-4c89-8b4c-ba87af80d573 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:55.808156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9ac68-a647-4c89-8b4c-ba87af80d573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:55.827684 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-34c9ac68-a647-4c89-8b4c-ba87af80d573 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:55.828916 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-34c9ac68-a647-4c89-8b4c-ba87af80d573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:55.838079 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1302/2634] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:18:55.844192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e467a2-cb41-4bfd-acee-5dbc33a0b1ed None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:55.854368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e467a2-cb41-4bfd-acee-5dbc33a0b1ed None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:55.859602 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a81730-d664-426c-bfdf-7c4830754ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:55.860300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a81730-d664-426c-bfdf-7c4830754ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:55.860656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a81730-d664-426c-bfdf-7c4830754ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:55.860922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a81730-d664-426c-bfdf-7c4830754ee8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:55.861661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a81730-d664-426c-bfdf-7c4830754ee8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:18:55.861970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a81730-d664-426c-bfdf-7c4830754ee8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:55.863644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a81730-d664-426c-bfdf-7c4830754ee8 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:55.875720 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1333/2635] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:18:55.880246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9ac68-a647-4c89-8b4c-ba87af80d573 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:55.890096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9ac68-a647-4c89-8b4c-ba87af80d573 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:55.896022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706276e2-0316-49f6-b829-eac8dbabb7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:55.896689 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706276e2-0316-49f6-b829-eac8dbabb7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:55.896976 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706276e2-0316-49f6-b829-eac8dbabb7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:55.897254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706276e2-0316-49f6-b829-eac8dbabb7a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:55.897977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706276e2-0316-49f6-b829-eac8dbabb7a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:55.900088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706276e2-0316-49f6-b829-eac8dbabb7a9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:55.932755 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1303/2636] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:55 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:55.937366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a81730-d664-426c-bfdf-7c4830754ee8 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:55.948163 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a81730-d664-426c-bfdf-7c4830754ee8 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:56.011878 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ba58f-6ec8-4f4d-b33a-fcb7e3f40d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:56.012963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ba58f-6ec8-4f4d-b33a-fcb7e3f40d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.013419 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ba58f-6ec8-4f4d-b33a-fcb7e3f40d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.013846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ba58f-6ec8-4f4d-b33a-fcb7e3f40d8b admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.015103 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ba58f-6ec8-4f4d-b33a-fcb7e3f40d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:56.017895 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ba58f-6ec8-4f4d-b33a-fcb7e3f40d8b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:56.045186 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1334/2637] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:55 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:18:56.049626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706276e2-0316-49f6-b829-eac8dbabb7a9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:56.060732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706276e2-0316-49f6-b829-eac8dbabb7a9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:56.065979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cf21f-45f4-4dba-b6d8-85bad2ba7e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:56.066789 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf21f-45f4-4dba-b6d8-85bad2ba7e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.067075 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf21f-45f4-4dba-b6d8-85bad2ba7e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.067357 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf21f-45f4-4dba-b6d8-85bad2ba7e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.068102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cf21f-45f4-4dba-b6d8-85bad2ba7e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:56.069746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cf21f-45f4-4dba-b6d8-85bad2ba7e88 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:56.080424 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1304/2638] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:56.084476 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ba58f-6ec8-4f4d-b33a-fcb7e3f40d8b admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:56.095714 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ba58f-6ec8-4f4d-b33a-fcb7e3f40d8b admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:56.101205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e5d42-5f42-4484-911d-efd3907b0c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:56.101871 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e5d42-5f42-4484-911d-efd3907b0c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.102155 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e5d42-5f42-4484-911d-efd3907b0c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.102471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e5d42-5f42-4484-911d-efd3907b0c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.103192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e5d42-5f42-4484-911d-efd3907b0c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:56.104796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e5d42-5f42-4484-911d-efd3907b0c02 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:56.115145 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1335/2639] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:56.119194 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cf21f-45f4-4dba-b6d8-85bad2ba7e88 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:56.129642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cf21f-45f4-4dba-b6d8-85bad2ba7e88 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:56.135389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13503f-abb1-43ff-a460-d5659478dfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:56.136275 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13503f-abb1-43ff-a460-d5659478dfb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.136560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13503f-abb1-43ff-a460-d5659478dfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.136847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13503f-abb1-43ff-a460-d5659478dfb1 admin admin] PATH_INFO: `/v3/projects/5e00f60c5cc6497c894a39b2f09251cf/users/353e2d4b931642dea7998944dc5fbf46/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.152400 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13503f-abb1-43ff-a460-d5659478dfb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e00f60c5cc6497c894a39b2f09251cf, user_id=353e2d4b931642dea7998944dc5fbf46, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:56.155156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13503f-abb1-43ff-a460-d5659478dfb1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:56.171871 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1305/2640] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:56 2022] PUT /identity/v3/projects/5e00f60c5cc6497c894a39b2f09251cf/users/353e2d4b931642dea7998944dc5fbf46/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:56.184269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74c0f5-1559-46b9-9dd0-0ba37f3f0df7 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.184547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74c0f5-1559-46b9-9dd0-0ba37f3f0df7 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.184826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74c0f5-1559-46b9-9dd0-0ba37f3f0df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.214197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-0f74c0f5-1559-46b9-9dd0-0ba37f3f0df7 None None] MFA Rules not processed for user `353e2d4b931642dea7998944dc5fbf46`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:56.215435 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0f74c0f5-1559-46b9-9dd0-0ba37f3f0df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:56.250462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:56.282832 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1336/2641] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:56.287576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13503f-abb1-43ff-a460-d5659478dfb1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:56.349626 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13503f-abb1-43ff-a460-d5659478dfb1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:56.355485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060c999-76c2-4d0d-8385-812fb39e2f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:56.356216 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c999-76c2-4d0d-8385-812fb39e2f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.356497 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c999-76c2-4d0d-8385-812fb39e2f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.356772 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c999-76c2-4d0d-8385-812fb39e2f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.357471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060c999-76c2-4d0d-8385-812fb39e2f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:56.359492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060c999-76c2-4d0d-8385-812fb39e2f75 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:56.363338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:56.390900 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1306/2642] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:18:56 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:56.394993 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74c0f5-1559-46b9-9dd0-0ba37f3f0df7 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:56.405866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74c0f5-1559-46b9-9dd0-0ba37f3f0df7 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:56.468352 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de146594-1f3e-4627-8ac4-2be80daa57e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:56.469037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de146594-1f3e-4627-8ac4-2be80daa57e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.469362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de146594-1f3e-4627-8ac4-2be80daa57e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.469736 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de146594-1f3e-4627-8ac4-2be80daa57e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.470874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de146594-1f3e-4627-8ac4-2be80daa57e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:56.473316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de146594-1f3e-4627-8ac4-2be80daa57e1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:56.497764 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1337/2643] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:18:56 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:18:56.502153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060c999-76c2-4d0d-8385-812fb39e2f75 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:56.514192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060c999-76c2-4d0d-8385-812fb39e2f75 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:56.520156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3de3e2-3cdf-47f5-a524-ad0abd0449a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:56.520829 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3de3e2-3cdf-47f5-a524-ad0abd0449a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.521112 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3de3e2-3cdf-47f5-a524-ad0abd0449a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.521390 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3de3e2-3cdf-47f5-a524-ad0abd0449a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.522136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3de3e2-3cdf-47f5-a524-ad0abd0449a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:56.523813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3de3e2-3cdf-47f5-a524-ad0abd0449a0 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:56.535752 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1307/2644] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:56.540332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de146594-1f3e-4627-8ac4-2be80daa57e1 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:56.552573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de146594-1f3e-4627-8ac4-2be80daa57e1 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:56.557816 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7b830-2ea5-4606-86ce-a9e026f5e7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:56.558757 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7b830-2ea5-4606-86ce-a9e026f5e7ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.559130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7b830-2ea5-4606-86ce-a9e026f5e7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.559407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7b830-2ea5-4606-86ce-a9e026f5e7ba admin admin] PATH_INFO: `/v3/projects/51ec8c9ff40448659ddd4438c3cd19da/users/138af37ceac04362880bce20d0f5f5aa/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.574121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7b830-2ea5-4606-86ce-a9e026f5e7ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ec8c9ff40448659ddd4438c3cd19da, user_id=138af37ceac04362880bce20d0f5f5aa, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:56.576936 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7b830-2ea5-4606-86ce-a9e026f5e7ba admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:56.591744 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1338/2645] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:56 2022] PUT /identity/v3/projects/51ec8c9ff40448659ddd4438c3cd19da/users/138af37ceac04362880bce20d0f5f5aa/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:56.595671 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3de3e2-3cdf-47f5-a524-ad0abd0449a0 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:56.605699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3de3e2-3cdf-47f5-a524-ad0abd0449a0 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:56.660886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2684d35-4f7d-4edb-afa9-6c8dbe8372f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:56.661549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2684d35-4f7d-4edb-afa9-6c8dbe8372f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.661832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2684d35-4f7d-4edb-afa9-6c8dbe8372f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.662104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2684d35-4f7d-4edb-afa9-6c8dbe8372f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.663081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2684d35-4f7d-4edb-afa9-6c8dbe8372f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:56.664827 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2684d35-4f7d-4edb-afa9-6c8dbe8372f7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:56.675063 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1308/2646] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:18:56.680095 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7b830-2ea5-4606-86ce-a9e026f5e7ba admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:56.692517 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7b830-2ea5-4606-86ce-a9e026f5e7ba admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:56.697780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816adaa-15b3-4852-8bc8-ed31b6298f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:56.698718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816adaa-15b3-4852-8bc8-ed31b6298f75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.698987 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816adaa-15b3-4852-8bc8-ed31b6298f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.699261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816adaa-15b3-4852-8bc8-ed31b6298f75 admin admin] PATH_INFO: `/v3/projects/51ec8c9ff40448659ddd4438c3cd19da/users/138af37ceac04362880bce20d0f5f5aa/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.701327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816adaa-15b3-4852-8bc8-ed31b6298f75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ec8c9ff40448659ddd4438c3cd19da, user_id=138af37ceac04362880bce20d0f5f5aa, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:56.704051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816adaa-15b3-4852-8bc8-ed31b6298f75 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:56.715101 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1339/2647] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:18:56 2022] PUT /identity/v3/projects/51ec8c9ff40448659ddd4438c3cd19da/users/138af37ceac04362880bce20d0f5f5aa/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:18:56.725740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e616c-158b-40fa-9533-9fffc1af2658 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.725949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e616c-158b-40fa-9533-9fffc1af2658 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.726248 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e616c-158b-40fa-9533-9fffc1af2658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.752632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-242e616c-158b-40fa-9533-9fffc1af2658 None None] MFA Rules not processed for user `138af37ceac04362880bce20d0f5f5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:18:56.753857 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-242e616c-158b-40fa-9533-9fffc1af2658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:56.825524 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1309/2648] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:18:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:18:56.837197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816adaa-15b3-4852-8bc8-ed31b6298f75 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:18:56.839724 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1816adaa-15b3-4852-8bc8-ed31b6298f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:18:56.901402 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816adaa-15b3-4852-8bc8-ed31b6298f75 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:18:56.906009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fe2e7-d6d4-468a-add8-6592ce2264f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:18:56.906687 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe2e7-d6d4-468a-add8-6592ce2264f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:18:56.906974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe2e7-d6d4-468a-add8-6592ce2264f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:18:56.907246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe2e7-d6d4-468a-add8-6592ce2264f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:18:56.948932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:56.968165 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fe2e7-d6d4-468a-add8-6592ce2264f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:18:56.969619 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fe2e7-d6d4-468a-add8-6592ce2264f9 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:18:56.979293 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1340/2649] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:18:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:19:01.244404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e616c-158b-40fa-9533-9fffc1af2658 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:01.246144 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-242e616c-158b-40fa-9533-9fffc1af2658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:01.261196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e616c-158b-40fa-9533-9fffc1af2658 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:01.265939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53683f1-e48d-459b-8795-d27765f17847 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:01.266643 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53683f1-e48d-459b-8795-d27765f17847 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:01.266926 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53683f1-e48d-459b-8795-d27765f17847 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:01.267200 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53683f1-e48d-459b-8795-d27765f17847 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:01.277045 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53683f1-e48d-459b-8795-d27765f17847 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:01.278538 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53683f1-e48d-459b-8795-d27765f17847 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:01.288342 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1310/2650] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:19:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:19:06.205154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fe2e7-d6d4-468a-add8-6592ce2264f9 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:06.272248 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fe2e7-d6d4-468a-add8-6592ce2264f9 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:06.277402 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:06.278316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:06.278599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:06.278870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] PATH_INFO: `/v3/users/353e2d4b931642dea7998944dc5fbf46` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:06.280101 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=353e2d4b931642dea7998944dc5fbf46)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:06.282051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:06.332328 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '353e2d4b931642dea7998944dc5fbf46'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:06.332901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '353e2d4b931642dea7998944dc5fbf46'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:06.346159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '353e2d4b931642dea7998944dc5fbf46'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:06.353796 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '353e2d4b931642dea7998944dc5fbf46'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:06.354533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '353e2d4b931642dea7998944dc5fbf46'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:06.368418 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1341/2651] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:19:06 2022] DELETE /identity/v3/users/353e2d4b931642dea7998944dc5fbf46 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:19:06.373063 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d7444-dc12-4eed-bf6f-8ea15424b4c2 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:06.373578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d7444-dc12-4eed-bf6f-8ea15424b4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:06.373700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d7444-dc12-4eed-bf6f-8ea15424b4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:06.391945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ff7d7444-dc12-4eed-bf6f-8ea15424b4c2 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:19:06.393203 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ff7d7444-dc12-4eed-bf6f-8ea15424b4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:06.433095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:06.455764 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1311/2652] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:19:06.466351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:06.468081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:06.537965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95262973-f59f-4c62-9bfb-b16fc0b89d54 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:06.541897 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd353796-7717-4cb8-b136-210999fbb071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:06.542812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd353796-7717-4cb8-b136-210999fbb071 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:06.543106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd353796-7717-4cb8-b136-210999fbb071 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:06.543387 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd353796-7717-4cb8-b136-210999fbb071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:06.553412 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd353796-7717-4cb8-b136-210999fbb071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:06.554907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd353796-7717-4cb8-b136-210999fbb071 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:06.564729 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1342/2653] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:19:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:19:06.844135 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d7444-dc12-4eed-bf6f-8ea15424b4c2 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:06.845847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ff7d7444-dc12-4eed-bf6f-8ea15424b4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:06.860035 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d7444-dc12-4eed-bf6f-8ea15424b4c2 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:06.866699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:06.867893 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:06.868213 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:06.868408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] PATH_INFO: `/v3/projects/5e00f60c5cc6497c894a39b2f09251cf` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:06.869737 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e00f60c5cc6497c894a39b2f09251cf)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:06.871686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:06.921807 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e00f60c5cc6497c894a39b2f09251cf', 'request_id': 'req-85c453e4-84f8-4b0b-9674-766694003b76'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:06.930605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e00f60c5cc6497c894a39b2f09251cf', 'request_id': 'req-85c453e4-84f8-4b0b-9674-766694003b76'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:06.935974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e00f60c5cc6497c894a39b2f09251cf', 'request_id': 'req-85c453e4-84f8-4b0b-9674-766694003b76'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:06.947132 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] The token cache is being invalidate because project 5e00f60c5cc6497c894a39b2f09251cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:19:06.947423 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:06.948094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:06.949745 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1312/2654] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:19:06 2022] DELETE /identity/v3/projects/5e00f60c5cc6497c894a39b2f09251cf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:19:06.955734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd353796-7717-4cb8-b136-210999fbb071 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:06.962162 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:06.969237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd353796-7717-4cb8-b136-210999fbb071 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:07.029195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23f059-5962-4aff-a5e4-b01690602ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:07.030071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23f059-5962-4aff-a5e4-b01690602ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:07.030382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23f059-5962-4aff-a5e4-b01690602ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:07.030653 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23f059-5962-4aff-a5e4-b01690602ba2 admin admin] PATH_INFO: `/v3/users/138af37ceac04362880bce20d0f5f5aa` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:07.031885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23f059-5962-4aff-a5e4-b01690602ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=138af37ceac04362880bce20d0f5f5aa)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:07.033846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23f059-5962-4aff-a5e4-b01690602ba2 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:07.082623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-cf23f059-5962-4aff-a5e4-b01690602ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138af37ceac04362880bce20d0f5f5aa'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:07.083296 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-cf23f059-5962-4aff-a5e4-b01690602ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138af37ceac04362880bce20d0f5f5aa'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:07.095327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-cf23f059-5962-4aff-a5e4-b01690602ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138af37ceac04362880bce20d0f5f5aa'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:07.102609 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-cf23f059-5962-4aff-a5e4-b01690602ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138af37ceac04362880bce20d0f5f5aa'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:07.103271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-cf23f059-5962-4aff-a5e4-b01690602ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138af37ceac04362880bce20d0f5f5aa'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:07.119102 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1343/2655] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:19:06 2022] DELETE /identity/v3/users/138af37ceac04362880bce20d0f5f5aa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:19:07.462672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:07.465104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:07.478434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c453e4-84f8-4b0b-9674-766694003b76 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:07.542483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:07.543376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:07.543654 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:07.543924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] PATH_INFO: `/v3/projects/51ec8c9ff40448659ddd4438c3cd19da` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:07.545279 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ec8c9ff40448659ddd4438c3cd19da)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:07.547237 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:07.596407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ec8c9ff40448659ddd4438c3cd19da', 'request_id': 'req-e8efd3d8-f811-46ee-b962-bc39629cc650'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:07.605711 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ec8c9ff40448659ddd4438c3cd19da', 'request_id': 'req-e8efd3d8-f811-46ee-b962-bc39629cc650'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:07.611609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ec8c9ff40448659ddd4438c3cd19da', 'request_id': 'req-e8efd3d8-f811-46ee-b962-bc39629cc650'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:07.613486 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:07.624147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] The token cache is being invalidate because project 51ec8c9ff40448659ddd4438c3cd19da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:19:07.624499 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:07.625416 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-e8efd3d8-f811-46ee-b962-bc39629cc650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:07.627270 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1313/2656] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:19:07 2022] DELETE /identity/v3/projects/51ec8c9ff40448659ddd4438c3cd19da => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:19:07.634667 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883fac8-7211-4130-8a39-8e8f3d37d537 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:07.635191 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883fac8-7211-4130-8a39-8e8f3d37d537 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:07.635191 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883fac8-7211-4130-8a39-8e8f3d37d537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:07.654667 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a883fac8-7211-4130-8a39-8e8f3d37d537 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:19:07.655899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a883fac8-7211-4130-8a39-8e8f3d37d537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:07.717564 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1344/2657] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:19:07.727437 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bc57b-0373-410b-ad43-a619255faade None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:07.727797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bc57b-0373-410b-ad43-a619255faade None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:07.728076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bc57b-0373-410b-ad43-a619255faade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:07.747505 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-989bc57b-0373-410b-ad43-a619255faade None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:19:07.748740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-989bc57b-0373-410b-ad43-a619255faade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:07.757745 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1314/2658] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:19:07.762368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883fac8-7211-4130-8a39-8e8f3d37d537 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:07.772359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883fac8-7211-4130-8a39-8e8f3d37d537 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:07.777538 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a13321-8efc-4a31-95c0-e3b7e5e784ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:07.778196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a13321-8efc-4a31-95c0-e3b7e5e784ed admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:07.778478 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a13321-8efc-4a31-95c0-e3b7e5e784ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:07.778748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a13321-8efc-4a31-95c0-e3b7e5e784ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:07.779473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a13321-8efc-4a31-95c0-e3b7e5e784ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:19:07.779777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a13321-8efc-4a31-95c0-e3b7e5e784ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:07.781384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a13321-8efc-4a31-95c0-e3b7e5e784ed admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:07.792736 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1345/2659] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:19:07.797027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bc57b-0373-410b-ad43-a619255faade None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:07.806424 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bc57b-0373-410b-ad43-a619255faade None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:07.811410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ed857-6d41-46c2-8fd8-18fabf80de5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:07.812062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ed857-6d41-46c2-8fd8-18fabf80de5c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:07.812385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ed857-6d41-46c2-8fd8-18fabf80de5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:07.813685 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ed857-6d41-46c2-8fd8-18fabf80de5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:07.814432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ed857-6d41-46c2-8fd8-18fabf80de5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:07.816331 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ed857-6d41-46c2-8fd8-18fabf80de5c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:07.844316 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1315/2660] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:19:07 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:19:07.848636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a13321-8efc-4a31-95c0-e3b7e5e784ed admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:07.859022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a13321-8efc-4a31-95c0-e3b7e5e784ed admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:07.863791 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:07.917413 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de559806-69c5-4086-b869-b8797a1a52c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:07.918081 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de559806-69c5-4086-b869-b8797a1a52c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:07.918410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de559806-69c5-4086-b869-b8797a1a52c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:07.918685 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de559806-69c5-4086-b869-b8797a1a52c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:07.919371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de559806-69c5-4086-b869-b8797a1a52c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:07.921280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de559806-69c5-4086-b869-b8797a1a52c2 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:07.943888 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1346/2661] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:19:07 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:19:07.948377 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ed857-6d41-46c2-8fd8-18fabf80de5c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:07.959136 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ed857-6d41-46c2-8fd8-18fabf80de5c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:07.964802 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27140a-9747-4a39-964b-8f35758224ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:07.965733 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27140a-9747-4a39-964b-8f35758224ce admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:07.966253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27140a-9747-4a39-964b-8f35758224ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:07.966766 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27140a-9747-4a39-964b-8f35758224ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:07.967524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27140a-9747-4a39-964b-8f35758224ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:07.969501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27140a-9747-4a39-964b-8f35758224ce admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:07.982037 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1316/2662] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:19:07.992661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de559806-69c5-4086-b869-b8797a1a52c2 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:08.003257 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de559806-69c5-4086-b869-b8797a1a52c2 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:08.008725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0f264-54df-4d8a-91e6-78c634687e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:08.009399 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0f264-54df-4d8a-91e6-78c634687e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:08.009684 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0f264-54df-4d8a-91e6-78c634687e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:08.009966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0f264-54df-4d8a-91e6-78c634687e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:08.010731 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0f264-54df-4d8a-91e6-78c634687e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:08.012376 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0f264-54df-4d8a-91e6-78c634687e90 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:08.022441 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1347/2663] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:19:08.026525 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27140a-9747-4a39-964b-8f35758224ce admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:08.037098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27140a-9747-4a39-964b-8f35758224ce admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:08.043094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd54c5e-9d41-476b-8efa-ae035c4d1c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:08.043993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd54c5e-9d41-476b-8efa-ae035c4d1c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:08.044280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd54c5e-9d41-476b-8efa-ae035c4d1c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:08.044563 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd54c5e-9d41-476b-8efa-ae035c4d1c5e admin admin] PATH_INFO: `/v3/projects/6d0bea7d04f74ca285a6ed4178f893e1/users/674d5258ec2a4b80b360beae7e106e8c/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:08.060408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd54c5e-9d41-476b-8efa-ae035c4d1c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0bea7d04f74ca285a6ed4178f893e1, user_id=674d5258ec2a4b80b360beae7e106e8c, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:08.063124 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd54c5e-9d41-476b-8efa-ae035c4d1c5e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:08.078358 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1317/2664] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:19:08 2022] PUT /identity/v3/projects/6d0bea7d04f74ca285a6ed4178f893e1/users/674d5258ec2a4b80b360beae7e106e8c/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:19:08.090422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb079b-3ff2-46a7-9c1c-87a7ac530eb8 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:08.090715 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb079b-3ff2-46a7-9c1c-87a7ac530eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:08.090978 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb079b-3ff2-46a7-9c1c-87a7ac530eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:08.118352 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-6ddb079b-3ff2-46a7-9c1c-87a7ac530eb8 None None] MFA Rules not processed for user `674d5258ec2a4b80b360beae7e106e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:19:08.119567 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6ddb079b-3ff2-46a7-9c1c-87a7ac530eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:08.185501 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1348/2665] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:19:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:19:08.197549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd54c5e-9d41-476b-8efa-ae035c4d1c5e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:08.199469 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5dd54c5e-9d41-476b-8efa-ae035c4d1c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:08.260844 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd54c5e-9d41-476b-8efa-ae035c4d1c5e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:08.265881 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ae0ca-e1b7-4512-8e89-9d7c7ad86dab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:08.266585 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ae0ca-e1b7-4512-8e89-9d7c7ad86dab service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:08.267033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ae0ca-e1b7-4512-8e89-9d7c7ad86dab service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:08.267319 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ae0ca-e1b7-4512-8e89-9d7c7ad86dab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:08.276915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ae0ca-e1b7-4512-8e89-9d7c7ad86dab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:08.278421 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ae0ca-e1b7-4512-8e89-9d7c7ad86dab service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:08.289923 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1318/2666] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:19:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:19:10.870812 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb079b-3ff2-46a7-9c1c-87a7ac530eb8 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:10.935395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb079b-3ff2-46a7-9c1c-87a7ac530eb8 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:10.940269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:10.941160 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:10.941449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:10.941723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] PATH_INFO: `/v3/users/674d5258ec2a4b80b360beae7e106e8c` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:10.942979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=674d5258ec2a4b80b360beae7e106e8c)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:10.944995 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:10.966352 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:10.997276 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '674d5258ec2a4b80b360beae7e106e8c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:10.997977 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '674d5258ec2a4b80b360beae7e106e8c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:11.011272 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '674d5258ec2a4b80b360beae7e106e8c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:11.017890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '674d5258ec2a4b80b360beae7e106e8c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:11.018585 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '674d5258ec2a4b80b360beae7e106e8c'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:11.034861 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1349/2667] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:19:10 2022] DELETE /identity/v3/users/674d5258ec2a4b80b360beae7e106e8c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:19:11.039921 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9dfa5-2aea-4a7a-9d79-cd8347bd3c6c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:11.040478 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9dfa5-2aea-4a7a-9d79-cd8347bd3c6c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:11.040478 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9dfa5-2aea-4a7a-9d79-cd8347bd3c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:11.058728 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fee9dfa5-2aea-4a7a-9d79-cd8347bd3c6c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:19:11.059944 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fee9dfa5-2aea-4a7a-9d79-cd8347bd3c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:11.125972 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1319/2668] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:19:11.138779 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:11.141114 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:11.203750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b7b28-203c-4f97-8f1d-9afd922238f6 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:11.207959 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ac951-f647-429b-b531-9cfeda2f5758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:11.208617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ac951-f647-429b-b531-9cfeda2f5758 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:11.208909 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ac951-f647-429b-b531-9cfeda2f5758 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:11.209192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ac951-f647-429b-b531-9cfeda2f5758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:11.219205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ac951-f647-429b-b531-9cfeda2f5758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:11.220693 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ac951-f647-429b-b531-9cfeda2f5758 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:11.231165 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1350/2669] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:19:11.750276 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9dfa5-2aea-4a7a-9d79-cd8347bd3c6c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:11.751821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fee9dfa5-2aea-4a7a-9d79-cd8347bd3c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:11.765314 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9dfa5-2aea-4a7a-9d79-cd8347bd3c6c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:11.770797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:11.771697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:11.771970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:11.772252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] PATH_INFO: `/v3/projects/6d0bea7d04f74ca285a6ed4178f893e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:11.773612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d0bea7d04f74ca285a6ed4178f893e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:11.775572 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:11.812350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:11.825660 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d0bea7d04f74ca285a6ed4178f893e1', 'request_id': 'req-cac7b4cd-b590-49c0-8795-f515da370d97'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:11.833859 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d0bea7d04f74ca285a6ed4178f893e1', 'request_id': 'req-cac7b4cd-b590-49c0-8795-f515da370d97'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:11.839899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d0bea7d04f74ca285a6ed4178f893e1', 'request_id': 'req-cac7b4cd-b590-49c0-8795-f515da370d97'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:11.850699 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] The token cache is being invalidate because project 6d0bea7d04f74ca285a6ed4178f893e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:19:11.850983 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:11.851576 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:19:11.853129 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1320/2670] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:19:11 2022] DELETE /identity/v3/projects/6d0bea7d04f74ca285a6ed4178f893e1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:19:21.226054 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:DELETE=8,keystone:SELECT=46,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:21.847834 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=13,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:22.120645 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff25dd-4be1-4a3e-b392-881f9b7bdeb3 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:22.120932 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff25dd-4be1-4a3e-b392-881f9b7bdeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:22.121201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff25dd-4be1-4a3e-b392-881f9b7bdeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:22.139670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-25ff25dd-4be1-4a3e-b392-881f9b7bdeb3 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:19:22.140780 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-25ff25dd-4be1-4a3e-b392-881f9b7bdeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:22.201292 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1351/2671] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:19:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:19:22.255508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:22.257016 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:22.327847 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7b4cd-b590-49c0-8795-f515da370d97 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:22.333892 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3340f7-cfc5-4ba5-a833-22ffd84e4472 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:22.334574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3340f7-cfc5-4ba5-a833-22ffd84e4472 service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:22.334860 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3340f7-cfc5-4ba5-a833-22ffd84e4472 service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:22.335137 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3340f7-cfc5-4ba5-a833-22ffd84e4472 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:22.343924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3340f7-cfc5-4ba5-a833-22ffd84e4472 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:22.345537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3340f7-cfc5-4ba5-a833-22ffd84e4472 service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:22.353636 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1321/2672] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:19:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:19:32.190734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=25 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:32.350597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=32 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:36.733167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbfc79-c6fc-489a-b354-236b7d3c3b82 None None] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:36.733716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbfc79-c6fc-489a-b354-236b7d3c3b82 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:36.733716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbfc79-c6fc-489a-b354-236b7d3c3b82 None None] PATH_INFO: `/` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:36.735455 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1352/2673] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jul 5 10:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 10:19:36.740508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18a54f-b4d9-4fb8-8631-625968ab8c75 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:36.740949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18a54f-b4d9-4fb8-8631-625968ab8c75 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:36.741037 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18a54f-b4d9-4fb8-8631-625968ab8c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:36.764129 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0b18a54f-b4d9-4fb8-8631-625968ab8c75 None None] MFA Rules not processed for user `fee6ee8f99c34aa281bf02ecd3f23216`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:19:36.765364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0b18a54f-b4d9-4fb8-8631-625968ab8c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:36.837399 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1322/2674] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jul 5 10:19:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:19:36.849205 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbfc79-c6fc-489a-b354-236b7d3c3b82 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:36.850924 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5efbfc79-c6fc-489a-b354-236b7d3c3b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:36.908011 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbfc79-c6fc-489a-b354-236b7d3c3b82 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:36.912028 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7ebe9-8c6d-4370-bf2e-298cdf1432ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b97736e49e2a4bfb9cf7093cfa8b2b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:36.912676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7ebe9-8c6d-4370-bf2e-298cdf1432ae service glance] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:36.912957 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7ebe9-8c6d-4370-bf2e-298cdf1432ae service glance] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:36.913233 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7ebe9-8c6d-4370-bf2e-298cdf1432ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:36.923890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7ebe9-8c6d-4370-bf2e-298cdf1432ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:36.925332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7ebe9-8c6d-4370-bf2e-298cdf1432ae service glance] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:36.936087 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1353/2675] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:19:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:19:38.759001 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc486d8-c7dc-4780-a5bc-d3d1492e0223 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:38.759528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc486d8-c7dc-4780-a5bc-d3d1492e0223 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:38.759528 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc486d8-c7dc-4780-a5bc-d3d1492e0223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:38.778391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-0fc486d8-c7dc-4780-a5bc-d3d1492e0223 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:19:38.779639 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0fc486d8-c7dc-4780-a5bc-d3d1492e0223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:38.787902 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1323/2676] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:19:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:19:38.799838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7ebe9-8c6d-4370-bf2e-298cdf1432ae service glance] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:38.811660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7ebe9-8c6d-4370-bf2e-298cdf1432ae service glance] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:38.817934 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52803428-435b-4f86-83a9-e52a9703f9a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:38.818437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52803428-435b-4f86-83a9-e52a9703f9a8 service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:38.818717 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52803428-435b-4f86-83a9-e52a9703f9a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:38.818992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52803428-435b-4f86-83a9-e52a9703f9a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:38.828258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52803428-435b-4f86-83a9-e52a9703f9a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:38.829774 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52803428-435b-4f86-83a9-e52a9703f9a8 service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:38.839439 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1354/2677] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:19:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 05 10:19:45.691225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc486d8-c7dc-4780-a5bc-d3d1492e0223 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:45.692897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0fc486d8-c7dc-4780-a5bc-d3d1492e0223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:45.711534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc486d8-c7dc-4780-a5bc-d3d1492e0223 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:45.718420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed538e89-2242-43fd-b097-e5f7cbc9037f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:45.719250 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed538e89-2242-43fd-b097-e5f7cbc9037f service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:45.719579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed538e89-2242-43fd-b097-e5f7cbc9037f service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:45.719923 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed538e89-2242-43fd-b097-e5f7cbc9037f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:45.721817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ed538e89-2242-43fd-b097-e5f7cbc9037f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:45.783494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed538e89-2242-43fd-b097-e5f7cbc9037f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:45.785211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed538e89-2242-43fd-b097-e5f7cbc9037f service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:45.795247 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1324/2678] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:19:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:19:48.835763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=32 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:55.790353 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=62 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:58.724995 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52803428-435b-4f86-83a9-e52a9703f9a8 service swift] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:19:58.726616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-52803428-435b-4f86-83a9-e52a9703f9a8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:58.787951 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52803428-435b-4f86-83a9-e52a9703f9a8 service swift] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:19:58.792045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9283d7-5dae-4a7a-a045-d806af910293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:19:58.792685 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9283d7-5dae-4a7a-a045-d806af910293 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:19:58.792963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9283d7-5dae-4a7a-a045-d806af910293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:19:58.793232 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9283d7-5dae-4a7a-a045-d806af910293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:19:58.795234 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4d9283d7-5dae-4a7a-a045-d806af910293 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:19:58.854186 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9283d7-5dae-4a7a-a045-d806af910293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:19:58.855837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9283d7-5dae-4a7a-a045-d806af910293 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:19:58.865148 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1355/2679] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:19:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:20:01.923678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11542bc7-11cb-4889-bfee-24a6f1d5c98a None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:20:01.924199 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11542bc7-11cb-4889-bfee-24a6f1d5c98a None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:20:01.924199 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11542bc7-11cb-4889-bfee-24a6f1d5c98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:20:01.943757 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-11542bc7-11cb-4889-bfee-24a6f1d5c98a None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:20:01.945713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-11542bc7-11cb-4889-bfee-24a6f1d5c98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:20:01.953433 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1325/2680] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jul 5 10:20:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:20:01.967258 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9283d7-5dae-4a7a-a045-d806af910293 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:20:01.979251 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9283d7-5dae-4a7a-a045-d806af910293 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:20:01.984406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b896b-0242-4725-b2b3-a14eb8e54a20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:20:01.985054 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b896b-0242-4725-b2b3-a14eb8e54a20 service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:20:01.985333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b896b-0242-4725-b2b3-a14eb8e54a20 service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:20:01.985611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b896b-0242-4725-b2b3-a14eb8e54a20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:20:01.993917 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b896b-0242-4725-b2b3-a14eb8e54a20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:20:01.995598 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b896b-0242-4725-b2b3-a14eb8e54a20 service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:20:02.003935 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1356/2681] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:20:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:20:05.249231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11542bc7-11cb-4889-bfee-24a6f1d5c98a None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:20:05.262250 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11542bc7-11cb-4889-bfee-24a6f1d5c98a None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:20:05.266579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24880194-912f-490b-84c6-5d80b3fb4c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:20:05.267447 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24880194-912f-490b-84c6-5d80b3fb4c9f service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:20:05.267797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24880194-912f-490b-84c6-5d80b3fb4c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:20:05.268160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24880194-912f-490b-84c6-5d80b3fb4c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:20:05.270043 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-24880194-912f-490b-84c6-5d80b3fb4c9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:20:05.344625 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24880194-912f-490b-84c6-5d80b3fb4c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:20:05.346565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24880194-912f-490b-84c6-5d80b3fb4c9f service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:20:05.358248 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1326/2682] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:20:06.864347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc42994-057d-4973-a870-9cc5db947e51 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:20:06.865325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc42994-057d-4973-a870-9cc5db947e51 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:20:06.865703 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc42994-057d-4973-a870-9cc5db947e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:20:06.889249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-5dc42994-057d-4973-a870-9cc5db947e51 None None] MFA Rules not processed for user `ae9cd6d6d5824aa1932d3353578285af`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:20:06.890740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5dc42994-057d-4973-a870-9cc5db947e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:20:06.898303 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1357/2683] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:20:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:20:07.633491 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24880194-912f-490b-84c6-5d80b3fb4c9f service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:20:07.648085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24880194-912f-490b-84c6-5d80b3fb4c9f service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:20:07.652744 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04720ca5-1433-4d18-a941-d046e2291392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:20:07.653717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04720ca5-1433-4d18-a941-d046e2291392 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:20:07.654110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04720ca5-1433-4d18-a941-d046e2291392 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:20:07.654258 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04720ca5-1433-4d18-a941-d046e2291392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:20:07.656466 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-04720ca5-1433-4d18-a941-d046e2291392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:20:07.669674 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04720ca5-1433-4d18-a941-d046e2291392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:20:07.671268 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04720ca5-1433-4d18-a941-d046e2291392 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:20:07.680673 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1327/2684] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:20:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:20:08.909459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc42994-057d-4973-a870-9cc5db947e51 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:20:08.910980 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5dc42994-057d-4973-a870-9cc5db947e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:20:09.015091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc42994-057d-4973-a870-9cc5db947e51 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:20:09.024867 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1e0b2-3753-46cc-80ed-a546c19a041d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:20:09.025579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1e0b2-3753-46cc-80ed-a546c19a041d service placement] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:20:09.025865 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1e0b2-3753-46cc-80ed-a546c19a041d service placement] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:20:09.026141 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1e0b2-3753-46cc-80ed-a546c19a041d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:20:09.028667 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c4e1e0b2-3753-46cc-80ed-a546c19a041d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:20:09.055708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1e0b2-3753-46cc-80ed-a546c19a041d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:20:09.056928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1e0b2-3753-46cc-80ed-a546c19a041d service placement] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:20:09.068222 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1358/2685] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:20:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:20:13.283803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04720ca5-1433-4d18-a941-d046e2291392 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:20:13.285457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-04720ca5-1433-4d18-a941-d046e2291392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:20:13.302369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04720ca5-1433-4d18-a941-d046e2291392 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:20:13.307752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:20:13.309271 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:20:13.309863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:20:13.310422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] PATH_INFO: `/v3/users/9966a96281b745e684d0cd16bb470e6a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:20:13.312434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9966a96281b745e684d0cd16bb470e6a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:20:13.315062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:20:13.367168 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9966a96281b745e684d0cd16bb470e6a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.367781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9966a96281b745e684d0cd16bb470e6a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.368421 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9966a96281b745e684d0cd16bb470e6a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.375278 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9966a96281b745e684d0cd16bb470e6a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.386894 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9966a96281b745e684d0cd16bb470e6a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.402255 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1328/2686] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:20:13 2022] DELETE /identity/v3/users/9966a96281b745e684d0cd16bb470e6a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:20:13.582858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1e0b2-3753-46cc-80ed-a546c19a041d service placement] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:20:13.595923 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1e0b2-3753-46cc-80ed-a546c19a041d service placement] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:20:13.622191 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=100 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:13.665964 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:20:13.666911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:20:13.667195 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:20:13.667540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] PATH_INFO: `/v3/projects/815e83f886cc4c78bcdc31eb4230c782` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:20:13.668770 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=815e83f886cc4c78bcdc31eb4230c782)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:20:13.671233 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:20:13.718734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '815e83f886cc4c78bcdc31eb4230c782', 'request_id': 'req-a6390931-d9e8-4ade-baf7-824e5afb1741'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.724385 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '815e83f886cc4c78bcdc31eb4230c782', 'request_id': 'req-a6390931-d9e8-4ade-baf7-824e5afb1741'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.730829 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '815e83f886cc4c78bcdc31eb4230c782', 'request_id': 'req-a6390931-d9e8-4ade-baf7-824e5afb1741'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.744558 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] The token cache is being invalidate because project 815e83f886cc4c78bcdc31eb4230c782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:20:13.744843 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.745635 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.747109 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1359/2687] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:20:13 2022] DELETE /identity/v3/projects/815e83f886cc4c78bcdc31eb4230c782 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:20:13.751393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:20:13.752851 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:20:13.766751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5433fd9-bd49-4489-abc7-01693b0d9bdb admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:20:13.812129 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:13.827743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:20:13.828638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:20:13.828933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:20:13.829252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] PATH_INFO: `/v3/users/721521f50380490ea84fb46734f0f592` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:20:13.830490 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] RBAC: Authorizing `identity:delete_user(user_id=721521f50380490ea84fb46734f0f592)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:20:13.832415 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:20:13.881970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '721521f50380490ea84fb46734f0f592'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.882560 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '721521f50380490ea84fb46734f0f592'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.883149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '721521f50380490ea84fb46734f0f592'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.889876 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '721521f50380490ea84fb46734f0f592'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.901487 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '721521f50380490ea84fb46734f0f592'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:13.916288 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1329/2688] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:20:13 2022] DELETE /identity/v3/users/721521f50380490ea84fb46734f0f592 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:20:14.099848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:20:14.101461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:20:14.116768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6390931-d9e8-4ade-baf7-824e5afb1741 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:20:14.180447 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:20:14.181322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:20:14.181600 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:20:14.181883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] PATH_INFO: `/v3/projects/cd55aaf5ed51497bb395c22072a4bbd9` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:20:14.183070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd55aaf5ed51497bb395c22072a4bbd9)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:20:14.184973 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:20:14.246280 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd55aaf5ed51497bb395c22072a4bbd9', 'request_id': 'req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:14.251045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:14.252608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd55aaf5ed51497bb395c22072a4bbd9', 'request_id': 'req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:14.260027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd55aaf5ed51497bb395c22072a4bbd9', 'request_id': 'req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:14.274176 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] The token cache is being invalidate because project cd55aaf5ed51497bb395c22072a4bbd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:20:14.274503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:14.275115 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:20:14.276658 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1360/2689] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:20:14 2022] DELETE /identity/v3/projects/cd55aaf5ed51497bb395c22072a4bbd9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:20:23.912544 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:24.271244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=5,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:01.436680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:01.438495 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:01.514473 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039fcfe-abea-48c0-9a7d-1758b045007d admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:01.577358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cdb63a-ce86-4577-9781-3b5a1952ca5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:01.577934 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cdb63a-ce86-4577-9781-3b5a1952ca5a service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:01.578285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cdb63a-ce86-4577-9781-3b5a1952ca5a service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:01.578566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cdb63a-ce86-4577-9781-3b5a1952ca5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:01.580461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-08cdb63a-ce86-4577-9781-3b5a1952ca5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:01.592970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cdb63a-ce86-4577-9781-3b5a1952ca5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:01.594755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cdb63a-ce86-4577-9781-3b5a1952ca5a service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:01.604752 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1330/2690] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:21:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:21:05.339520 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:05.341090 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:05.400203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbacefe-fba9-41a8-8cc9-a9aa12f391b7 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:05.404207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367a45e-cbf8-470d-a34b-d0c0d487e444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:05.404870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367a45e-cbf8-470d-a34b-d0c0d487e444 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:05.405159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367a45e-cbf8-470d-a34b-d0c0d487e444 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:05.405438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367a45e-cbf8-470d-a34b-d0c0d487e444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:05.407404 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f367a45e-cbf8-470d-a34b-d0c0d487e444 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:05.470965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367a45e-cbf8-470d-a34b-d0c0d487e444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:05.472420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367a45e-cbf8-470d-a34b-d0c0d487e444 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:05.481954 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1361/2691] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:21:08.961549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cdb63a-ce86-4577-9781-3b5a1952ca5a service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:08.963606 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-08cdb63a-ce86-4577-9781-3b5a1952ca5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:08.984494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cdb63a-ce86-4577-9781-3b5a1952ca5a service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:08.989052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3da702-b707-47d0-9119-bfb7ce9e1779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:08.990157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3da702-b707-47d0-9119-bfb7ce9e1779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:08.990499 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3da702-b707-47d0-9119-bfb7ce9e1779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:08.990768 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3da702-b707-47d0-9119-bfb7ce9e1779 admin admin] PATH_INFO: `/v3/users/51e20b9a489342d2b8f0f85542ab370c` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:08.997910 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3da702-b707-47d0-9119-bfb7ce9e1779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51e20b9a489342d2b8f0f85542ab370c)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:08.999928 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3da702-b707-47d0-9119-bfb7ce9e1779 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:09.050713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-da3da702-b707-47d0-9119-bfb7ce9e1779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51e20b9a489342d2b8f0f85542ab370c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:09.051367 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-da3da702-b707-47d0-9119-bfb7ce9e1779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51e20b9a489342d2b8f0f85542ab370c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:09.051961 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-da3da702-b707-47d0-9119-bfb7ce9e1779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51e20b9a489342d2b8f0f85542ab370c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:09.059399 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-da3da702-b707-47d0-9119-bfb7ce9e1779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51e20b9a489342d2b8f0f85542ab370c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:09.071502 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-da3da702-b707-47d0-9119-bfb7ce9e1779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51e20b9a489342d2b8f0f85542ab370c'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:09.086554 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1331/2692] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:21:08 2022] DELETE /identity/v3/users/51e20b9a489342d2b8f0f85542ab370c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:21:09.268907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367a45e-cbf8-470d-a34b-d0c0d487e444 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:09.280224 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367a45e-cbf8-470d-a34b-d0c0d487e444 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:09.339382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:09.340378 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:09.340678 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:09.340921 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] PATH_INFO: `/v3/projects/d343798527cf445295916cc8e8134bf3` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:09.342119 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d343798527cf445295916cc8e8134bf3)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:09.344066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:09.393124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd343798527cf445295916cc8e8134bf3', 'request_id': 'req-b2d0ca85-bded-46c7-942c-ef869100039f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:09.395136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:09.399640 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd343798527cf445295916cc8e8134bf3', 'request_id': 'req-b2d0ca85-bded-46c7-942c-ef869100039f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:09.405949 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd343798527cf445295916cc8e8134bf3', 'request_id': 'req-b2d0ca85-bded-46c7-942c-ef869100039f'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:09.418699 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] The token cache is being invalidate because project d343798527cf445295916cc8e8134bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:21:09.418987 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:09.419688 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-b2d0ca85-bded-46c7-942c-ef869100039f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:09.421346 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1362/2693] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:21:09 2022] DELETE /identity/v3/projects/d343798527cf445295916cc8e8134bf3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:21:09.426512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e91e58-7407-4ed1-a6b9-094a5ef7a0ec None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:09.426872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e91e58-7407-4ed1-a6b9-094a5ef7a0ec None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:09.427159 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e91e58-7407-4ed1-a6b9-094a5ef7a0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:09.435070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:09.447937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-55e91e58-7407-4ed1-a6b9-094a5ef7a0ec None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:09.449158 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-55e91e58-7407-4ed1-a6b9-094a5ef7a0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:09.511393 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1332/2694] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:09.519500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b90288-f5bd-4853-86cd-40ba819815ba None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:09.519794 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b90288-f5bd-4853-86cd-40ba819815ba None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:09.520080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b90288-f5bd-4853-86cd-40ba819815ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:09.538893 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-c7b90288-f5bd-4853-86cd-40ba819815ba None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:09.540097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c7b90288-f5bd-4853-86cd-40ba819815ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:09.548603 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1363/2695] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:09.552905 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e91e58-7407-4ed1-a6b9-094a5ef7a0ec None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:09.563020 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e91e58-7407-4ed1-a6b9-094a5ef7a0ec None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:09.568042 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c8e1f-8054-47f5-87ec-ba90a2988f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:09.568703 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c8e1f-8054-47f5-87ec-ba90a2988f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:09.568991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c8e1f-8054-47f5-87ec-ba90a2988f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:09.569259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c8e1f-8054-47f5-87ec-ba90a2988f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:09.569999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c8e1f-8054-47f5-87ec-ba90a2988f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:21:09.570335 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c8e1f-8054-47f5-87ec-ba90a2988f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:09.571956 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c8e1f-8054-47f5-87ec-ba90a2988f65 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:09.583782 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1333/2696] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:21:09.588277 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b90288-f5bd-4853-86cd-40ba819815ba None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:09.597540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b90288-f5bd-4853-86cd-40ba819815ba None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:09.603173 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd64f0d-386a-4027-982b-81ab8e712242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:09.603829 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd64f0d-386a-4027-982b-81ab8e712242 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:09.604110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd64f0d-386a-4027-982b-81ab8e712242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:09.604379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd64f0d-386a-4027-982b-81ab8e712242 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:09.605080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd64f0d-386a-4027-982b-81ab8e712242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:09.607039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd64f0d-386a-4027-982b-81ab8e712242 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:09.635991 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1364/2697] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:21:09 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:21:09.640323 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c8e1f-8054-47f5-87ec-ba90a2988f65 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:09.650249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c8e1f-8054-47f5-87ec-ba90a2988f65 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:09.707584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2635542-2015-486f-afcc-f63a28b52082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:09.708175 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2635542-2015-486f-afcc-f63a28b52082 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:09.708457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2635542-2015-486f-afcc-f63a28b52082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:09.708730 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2635542-2015-486f-afcc-f63a28b52082 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:09.709418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2635542-2015-486f-afcc-f63a28b52082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:09.711380 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2635542-2015-486f-afcc-f63a28b52082 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:09.733473 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1334/2698] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:21:09 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:21:09.737828 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd64f0d-386a-4027-982b-81ab8e712242 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:09.749318 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd64f0d-386a-4027-982b-81ab8e712242 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:09.754607 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddfc40-1abd-4cde-bf4e-6f2e27b98d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:09.755295 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddfc40-1abd-4cde-bf4e-6f2e27b98d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:09.755584 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddfc40-1abd-4cde-bf4e-6f2e27b98d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:09.755864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddfc40-1abd-4cde-bf4e-6f2e27b98d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:09.756599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddfc40-1abd-4cde-bf4e-6f2e27b98d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:09.758473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddfc40-1abd-4cde-bf4e-6f2e27b98d62 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:09.770822 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1365/2699] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:21:09.775623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2635542-2015-486f-afcc-f63a28b52082 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:09.789739 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2635542-2015-486f-afcc-f63a28b52082 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:09.795322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026882e4-0977-48ee-83ba-e26ec408a0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:09.795989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026882e4-0977-48ee-83ba-e26ec408a0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:09.796280 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026882e4-0977-48ee-83ba-e26ec408a0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:09.796554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026882e4-0977-48ee-83ba-e26ec408a0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:09.797296 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026882e4-0977-48ee-83ba-e26ec408a0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:09.798955 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026882e4-0977-48ee-83ba-e26ec408a0cd admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:09.809528 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1335/2700] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:21:09.813623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddfc40-1abd-4cde-bf4e-6f2e27b98d62 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:09.824475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddfc40-1abd-4cde-bf4e-6f2e27b98d62 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:09.830460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228851f-dced-4a09-8056-f0aca553b4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:09.831372 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228851f-dced-4a09-8056-f0aca553b4da admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:09.831648 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228851f-dced-4a09-8056-f0aca553b4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:09.831926 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228851f-dced-4a09-8056-f0aca553b4da admin admin] PATH_INFO: `/v3/projects/46ff9242d2b147f690786e1284d1cfef/users/a895ab25adee48a3851ad15657c07a96/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:09.847760 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228851f-dced-4a09-8056-f0aca553b4da admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ff9242d2b147f690786e1284d1cfef, user_id=a895ab25adee48a3851ad15657c07a96, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:09.850501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228851f-dced-4a09-8056-f0aca553b4da admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:09.865985 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1366/2701] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:21:09 2022] PUT /identity/v3/projects/46ff9242d2b147f690786e1284d1cfef/users/a895ab25adee48a3851ad15657c07a96/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:21:09.871229 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826aa93-9003-420f-8896-48a6636571b0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:09.871641 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826aa93-9003-420f-8896-48a6636571b0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:09.871929 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826aa93-9003-420f-8896-48a6636571b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:09.889607 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-9826aa93-9003-420f-8896-48a6636571b0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:09.890852 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9826aa93-9003-420f-8896-48a6636571b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:09.949461 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1336/2702] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:09.959485 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228851f-dced-4a09-8056-f0aca553b4da admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:09.961364 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1228851f-dced-4a09-8056-f0aca553b4da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:10.019623 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228851f-dced-4a09-8056-f0aca553b4da admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:10.023662 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85607e-dbc1-42ba-a297-e7035c0b4c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:10.024302 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85607e-dbc1-42ba-a297-e7035c0b4c95 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:10.024582 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85607e-dbc1-42ba-a297-e7035c0b4c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:10.024856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85607e-dbc1-42ba-a297-e7035c0b4c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:10.034617 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85607e-dbc1-42ba-a297-e7035c0b4c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:10.036080 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85607e-dbc1-42ba-a297-e7035c0b4c95 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:10.046168 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1367/2703] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:21:14.077285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b18e8-ee84-4a6b-b2e6-e36ae1b1f7d9 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:14.077805 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b18e8-ee84-4a6b-b2e6-e36ae1b1f7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:14.077906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b18e8-ee84-4a6b-b2e6-e36ae1b1f7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:14.098419 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:14.108172 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4e0b18e8-ee84-4a6b-b2e6-e36ae1b1f7d9 None None] MFA Rules not processed for user `a895ab25adee48a3851ad15657c07a96`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:14.109692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4e0b18e8-ee84-4a6b-b2e6-e36ae1b1f7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:14.175796 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1337/2704] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:14.190664 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85607e-dbc1-42ba-a297-e7035c0b4c95 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:14.192824 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3b85607e-dbc1-42ba-a297-e7035c0b4c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:14.271950 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85607e-dbc1-42ba-a297-e7035c0b4c95 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:14.277807 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48387505-c037-42d0-b7fe-9163d4a80dfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:14.278487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48387505-c037-42d0-b7fe-9163d4a80dfd service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:14.278768 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48387505-c037-42d0-b7fe-9163d4a80dfd service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:14.279037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48387505-c037-42d0-b7fe-9163d4a80dfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:14.286954 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48387505-c037-42d0-b7fe-9163d4a80dfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:14.288556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48387505-c037-42d0-b7fe-9163d4a80dfd service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:14.297527 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1368/2705] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:21:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:21:15.589016 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae1778-8e10-438e-9e28-78088402b816 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:15.589564 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae1778-8e10-438e-9e28-78088402b816 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:15.589564 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae1778-8e10-438e-9e28-78088402b816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:15.607227 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-17ae1778-8e10-438e-9e28-78088402b816 None None] MFA Rules not processed for user `a895ab25adee48a3851ad15657c07a96`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:15.608438 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-17ae1778-8e10-438e-9e28-78088402b816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:15.616508 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1338/2706] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:24.293838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:25.602196 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=30 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:51.003146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48387505-c037-42d0-b7fe-9163d4a80dfd service nova] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:51.015514 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48387505-c037-42d0-b7fe-9163d4a80dfd service nova] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:51.020063 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918f9fe-4b58-484e-b53e-0dbf261715d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:51.020707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918f9fe-4b58-484e-b53e-0dbf261715d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:51.020985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918f9fe-4b58-484e-b53e-0dbf261715d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:51.021257 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918f9fe-4b58-484e-b53e-0dbf261715d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:51.023452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0918f9fe-4b58-484e-b53e-0dbf261715d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:51.036887 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918f9fe-4b58-484e-b53e-0dbf261715d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:51.038394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918f9fe-4b58-484e-b53e-0dbf261715d4 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:51.048432 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1369/2707] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:21:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:21:54.202306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae1778-8e10-438e-9e28-78088402b816 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:54.204085 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-17ae1778-8e10-438e-9e28-78088402b816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:54.220033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae1778-8e10-438e-9e28-78088402b816 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:54.225368 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cb370-eb50-4277-a264-0d53a8b0bf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:54.226492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cb370-eb50-4277-a264-0d53a8b0bf1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:54.226784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cb370-eb50-4277-a264-0d53a8b0bf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:54.227068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cb370-eb50-4277-a264-0d53a8b0bf1b admin admin] PATH_INFO: `/v3/users/ae9cd6d6d5824aa1932d3353578285af` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:54.228381 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1cb370-eb50-4277-a264-0d53a8b0bf1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae9cd6d6d5824aa1932d3353578285af)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:54.230484 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1cb370-eb50-4277-a264-0d53a8b0bf1b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:54.283784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0b1cb370-eb50-4277-a264-0d53a8b0bf1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae9cd6d6d5824aa1932d3353578285af'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:54.284431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0b1cb370-eb50-4277-a264-0d53a8b0bf1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae9cd6d6d5824aa1932d3353578285af'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:54.285025 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0b1cb370-eb50-4277-a264-0d53a8b0bf1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae9cd6d6d5824aa1932d3353578285af'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:54.291765 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0b1cb370-eb50-4277-a264-0d53a8b0bf1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae9cd6d6d5824aa1932d3353578285af'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:54.303724 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-0b1cb370-eb50-4277-a264-0d53a8b0bf1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae9cd6d6d5824aa1932d3353578285af'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:54.318487 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1339/2708] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:21:54 2022] DELETE /identity/v3/users/ae9cd6d6d5824aa1932d3353578285af => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:21:54.516148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918f9fe-4b58-484e-b53e-0dbf261715d4 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:54.527536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918f9fe-4b58-484e-b53e-0dbf261715d4 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:54.591416 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:54.592616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:54.593147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:54.593491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] PATH_INFO: `/v3/projects/96cc7b005126403f94110029f4242a9f` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:54.595543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96cc7b005126403f94110029f4242a9f)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:54.598022 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:54.649839 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96cc7b005126403f94110029f4242a9f', 'request_id': 'req-a8a1b185-e63d-4779-a76d-0965e1f741a0'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:54.656266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96cc7b005126403f94110029f4242a9f', 'request_id': 'req-a8a1b185-e63d-4779-a76d-0965e1f741a0'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:54.662305 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96cc7b005126403f94110029f4242a9f', 'request_id': 'req-a8a1b185-e63d-4779-a76d-0965e1f741a0'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:54.675961 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] The token cache is being invalidate because project 96cc7b005126403f94110029f4242a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:21:54.676246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:54.676847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a8a1b185-e63d-4779-a76d-0965e1f741a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:54.678410 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1370/2709] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:21:54 2022] DELETE /identity/v3/projects/96cc7b005126403f94110029f4242a9f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:21:54.683672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e9b5a-1b37-48c8-b170-89dfb153a632 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:54.683969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e9b5a-1b37-48c8-b170-89dfb153a632 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:54.684253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e9b5a-1b37-48c8-b170-89dfb153a632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:54.703936 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4d6e9b5a-1b37-48c8-b170-89dfb153a632 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:54.705103 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4d6e9b5a-1b37-48c8-b170-89dfb153a632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:54.765159 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1340/2710] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:54.773097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659f181-99b4-4d6a-8bf8-8957170ef4ca None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:54.773381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659f181-99b4-4d6a-8bf8-8957170ef4ca None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:54.773654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659f181-99b4-4d6a-8bf8-8957170ef4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:54.793187 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e659f181-99b4-4d6a-8bf8-8957170ef4ca None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:54.794474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e659f181-99b4-4d6a-8bf8-8957170ef4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:54.803616 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1371/2711] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:54.808244 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e9b5a-1b37-48c8-b170-89dfb153a632 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:54.821481 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e9b5a-1b37-48c8-b170-89dfb153a632 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:54.827642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c55818-ee2a-4565-a28a-6e4e5531e7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:54.828338 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c55818-ee2a-4565-a28a-6e4e5531e7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:54.828624 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c55818-ee2a-4565-a28a-6e4e5531e7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:54.828907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c55818-ee2a-4565-a28a-6e4e5531e7a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:54.829650 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c55818-ee2a-4565-a28a-6e4e5531e7a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:21:54.829959 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c55818-ee2a-4565-a28a-6e4e5531e7a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:54.831630 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c55818-ee2a-4565-a28a-6e4e5531e7a9 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:54.845293 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1341/2712] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:21:54.849904 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659f181-99b4-4d6a-8bf8-8957170ef4ca None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:54.859740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659f181-99b4-4d6a-8bf8-8957170ef4ca None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:54.865735 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d5636-990b-4070-98df-0bf184f586fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:54.866440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d5636-990b-4070-98df-0bf184f586fc admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:54.866727 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d5636-990b-4070-98df-0bf184f586fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:54.867000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d5636-990b-4070-98df-0bf184f586fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:54.867706 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d5636-990b-4070-98df-0bf184f586fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:54.869661 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d5636-990b-4070-98df-0bf184f586fc admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:54.899364 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1372/2713] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:21:54 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:21:54.903419 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c55818-ee2a-4565-a28a-6e4e5531e7a9 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:54.914366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c55818-ee2a-4565-a28a-6e4e5531e7a9 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:54.973746 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a9ef7-e7a7-4ba2-89bd-7513fcd7891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:54.974479 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9ef7-e7a7-4ba2-89bd-7513fcd7891f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:54.974759 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9ef7-e7a7-4ba2-89bd-7513fcd7891f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:54.975040 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9ef7-e7a7-4ba2-89bd-7513fcd7891f admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:54.975771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a9ef7-e7a7-4ba2-89bd-7513fcd7891f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:54.977796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a9ef7-e7a7-4ba2-89bd-7513fcd7891f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:54.984658 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:55.001897 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1342/2714] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:21:54 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:21:55.006197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d5636-990b-4070-98df-0bf184f586fc admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:55.017055 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d5636-990b-4070-98df-0bf184f586fc admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:55.022576 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda73a0-3821-448b-a9c4-ebe501574d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:55.023384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda73a0-3821-448b-a9c4-ebe501574d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:55.023705 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda73a0-3821-448b-a9c4-ebe501574d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:55.024044 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda73a0-3821-448b-a9c4-ebe501574d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:55.024844 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda73a0-3821-448b-a9c4-ebe501574d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:55.026556 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda73a0-3821-448b-a9c4-ebe501574d5d admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:55.041662 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1373/2715] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:21:55.045725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a9ef7-e7a7-4ba2-89bd-7513fcd7891f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:55.056289 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a9ef7-e7a7-4ba2-89bd-7513fcd7891f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:55.061170 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc00a1-b18a-4b61-ba0f-6bea3016b146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:55.061899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc00a1-b18a-4b61-ba0f-6bea3016b146 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:55.062210 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc00a1-b18a-4b61-ba0f-6bea3016b146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:55.062539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc00a1-b18a-4b61-ba0f-6bea3016b146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:55.063303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc00a1-b18a-4b61-ba0f-6bea3016b146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:55.065030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc00a1-b18a-4b61-ba0f-6bea3016b146 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:55.075056 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1343/2716] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:21:55.079606 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda73a0-3821-448b-a9c4-ebe501574d5d admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:55.089621 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda73a0-3821-448b-a9c4-ebe501574d5d admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:55.094533 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef316-1b9d-4c8b-a121-6545fb1c1492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:55.095420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef316-1b9d-4c8b-a121-6545fb1c1492 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:55.095702 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef316-1b9d-4c8b-a121-6545fb1c1492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:55.095979 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef316-1b9d-4c8b-a121-6545fb1c1492 admin admin] PATH_INFO: `/v3/projects/f2a1304ab7a544d8b5d82bd39711eb3c/users/a09853f4dbc34f3781afbf61096531a3/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:55.110561 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edef316-1b9d-4c8b-a121-6545fb1c1492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a1304ab7a544d8b5d82bd39711eb3c, user_id=a09853f4dbc34f3781afbf61096531a3, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:55.113227 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edef316-1b9d-4c8b-a121-6545fb1c1492 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:55.128676 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1374/2717] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:21:55 2022] PUT /identity/v3/projects/f2a1304ab7a544d8b5d82bd39711eb3c/users/a09853f4dbc34f3781afbf61096531a3/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:21:55.139609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76cd86-f126-4880-a02e-62383bf3140f None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:55.139899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76cd86-f126-4880-a02e-62383bf3140f None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:55.140251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76cd86-f126-4880-a02e-62383bf3140f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:55.167886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ad76cd86-f126-4880-a02e-62383bf3140f None None] MFA Rules not processed for user `a09853f4dbc34f3781afbf61096531a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:55.169109 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ad76cd86-f126-4880-a02e-62383bf3140f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:55.251190 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1344/2718] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:55.263387 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef316-1b9d-4c8b-a121-6545fb1c1492 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:55.265767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1edef316-1b9d-4c8b-a121-6545fb1c1492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:55.282713 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:55.355985 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef316-1b9d-4c8b-a121-6545fb1c1492 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:55.361420 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b16b2-0360-4570-9da1-205fab369752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:55.362085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b16b2-0360-4570-9da1-205fab369752 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:55.362406 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b16b2-0360-4570-9da1-205fab369752 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:55.362685 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b16b2-0360-4570-9da1-205fab369752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:55.370616 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b16b2-0360-4570-9da1-205fab369752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:55.372249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b16b2-0360-4570-9da1-205fab369752 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:55.381093 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1375/2719] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:21:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:21:58.068134 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76cd86-f126-4880-a02e-62383bf3140f None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:58.140022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76cd86-f126-4880-a02e-62383bf3140f None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:58.145863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:58.146800 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:58.147049 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:58.147322 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] PATH_INFO: `/v3/users/a09853f4dbc34f3781afbf61096531a3` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:58.149331 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a09853f4dbc34f3781afbf61096531a3)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:58.152192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:58.211890 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a09853f4dbc34f3781afbf61096531a3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:58.212565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a09853f4dbc34f3781afbf61096531a3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:58.213208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a09853f4dbc34f3781afbf61096531a3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:58.220113 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a09853f4dbc34f3781afbf61096531a3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:58.230901 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:58.233388 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a09853f4dbc34f3781afbf61096531a3'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:58.248562 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1345/2720] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:21:58 2022] DELETE /identity/v3/users/a09853f4dbc34f3781afbf61096531a3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:21:58.253207 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516cbb2-6f56-4e9a-b626-1474d520afb0 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:58.253742 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516cbb2-6f56-4e9a-b626-1474d520afb0 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:58.253742 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516cbb2-6f56-4e9a-b626-1474d520afb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:58.271552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-4516cbb2-6f56-4e9a-b626-1474d520afb0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:58.272830 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4516cbb2-6f56-4e9a-b626-1474d520afb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:58.336101 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1376/2721] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:58.346930 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:58.348534 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:58.407062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f05611-a918-4cb8-90e0-eb3f8e546080 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:58.411784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2244519-90de-4e11-a47a-f9bc80bfd614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:58.412436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2244519-90de-4e11-a47a-f9bc80bfd614 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:58.412720 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2244519-90de-4e11-a47a-f9bc80bfd614 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:58.412994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2244519-90de-4e11-a47a-f9bc80bfd614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:58.423348 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2244519-90de-4e11-a47a-f9bc80bfd614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:58.424791 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2244519-90de-4e11-a47a-f9bc80bfd614 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:58.434927 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1346/2722] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:21:58.735868 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516cbb2-6f56-4e9a-b626-1474d520afb0 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:58.737316 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4516cbb2-6f56-4e9a-b626-1474d520afb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:58.749785 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516cbb2-6f56-4e9a-b626-1474d520afb0 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:58.756704 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:58.757639 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:58.757915 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:58.758201 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] PATH_INFO: `/v3/projects/f2a1304ab7a544d8b5d82bd39711eb3c` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:58.759574 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a1304ab7a544d8b5d82bd39711eb3c)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:58.761619 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:58.817656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a1304ab7a544d8b5d82bd39711eb3c', 'request_id': 'req-4a0704fb-278a-41c1-8952-82f8c253ea02'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:58.827374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a1304ab7a544d8b5d82bd39711eb3c', 'request_id': 'req-4a0704fb-278a-41c1-8952-82f8c253ea02'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:58.834272 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a1304ab7a544d8b5d82bd39711eb3c', 'request_id': 'req-4a0704fb-278a-41c1-8952-82f8c253ea02'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:58.849570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] The token cache is being invalidate because project f2a1304ab7a544d8b5d82bd39711eb3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:21:58.849851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:58.850500 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-4a0704fb-278a-41c1-8952-82f8c253ea02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:58.852084 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1377/2723] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:21:58 2022] DELETE /identity/v3/projects/f2a1304ab7a544d8b5d82bd39711eb3c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:21:58.857527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fc479-2071-4326-a65e-5dbb4cfe57be None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:58.857819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fc479-2071-4326-a65e-5dbb4cfe57be None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:58.858107 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fc479-2071-4326-a65e-5dbb4cfe57be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:58.877683 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-4c3fc479-2071-4326-a65e-5dbb4cfe57be None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:58.878963 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4c3fc479-2071-4326-a65e-5dbb4cfe57be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:58.939672 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1347/2724] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:58.947843 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4496d5-ede4-4df5-91a5-5f4a8914eda6 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:58.948139 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4496d5-ede4-4df5-91a5-5f4a8914eda6 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:58.948424 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4496d5-ede4-4df5-91a5-5f4a8914eda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:58.968074 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3f4496d5-ede4-4df5-91a5-5f4a8914eda6 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:58.969281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3f4496d5-ede4-4df5-91a5-5f4a8914eda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:58.978047 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1378/2725] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:58.982642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fc479-2071-4326-a65e-5dbb4cfe57be None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:58.993022 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fc479-2071-4326-a65e-5dbb4cfe57be None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:58.998680 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f19023-e678-45cd-beb7-532a491ac017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:58.999327 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19023-e678-45cd-beb7-532a491ac017 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:58.999610 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19023-e678-45cd-beb7-532a491ac017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:58.999881 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19023-e678-45cd-beb7-532a491ac017 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:59.000609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f19023-e678-45cd-beb7-532a491ac017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:21:59.000913 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f19023-e678-45cd-beb7-532a491ac017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:59.002525 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f19023-e678-45cd-beb7-532a491ac017 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:59.014196 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1348/2726] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:21:59.018789 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4496d5-ede4-4df5-91a5-5f4a8914eda6 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:59.028968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4496d5-ede4-4df5-91a5-5f4a8914eda6 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:59.034269 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c899c6-ac71-49fe-bf3a-c56097555e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:59.034974 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c899c6-ac71-49fe-bf3a-c56097555e90 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:59.035264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c899c6-ac71-49fe-bf3a-c56097555e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:59.035570 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c899c6-ac71-49fe-bf3a-c56097555e90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:59.036312 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c899c6-ac71-49fe-bf3a-c56097555e90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:59.038473 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c899c6-ac71-49fe-bf3a-c56097555e90 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:59.052664 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:59.068287 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1379/2727] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:21:59 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:21:59.072787 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f19023-e678-45cd-beb7-532a491ac017 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:59.083264 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f19023-e678-45cd-beb7-532a491ac017 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:59.144732 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4526c5e-66bb-4305-a530-b72d4d0318a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:59.145410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4526c5e-66bb-4305-a530-b72d4d0318a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:59.145696 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4526c5e-66bb-4305-a530-b72d4d0318a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:59.145974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4526c5e-66bb-4305-a530-b72d4d0318a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:59.146719 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4526c5e-66bb-4305-a530-b72d4d0318a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:59.148677 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4526c5e-66bb-4305-a530-b72d4d0318a5 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:59.169638 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1349/2728] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:21:59 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:21:59.175609 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c899c6-ac71-49fe-bf3a-c56097555e90 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:59.186746 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c899c6-ac71-49fe-bf3a-c56097555e90 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:59.191518 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e159c-ef95-48a1-b8b8-f615ef746f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:59.192175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e159c-ef95-48a1-b8b8-f615ef746f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:59.192479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e159c-ef95-48a1-b8b8-f615ef746f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:59.192775 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e159c-ef95-48a1-b8b8-f615ef746f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:59.193507 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e159c-ef95-48a1-b8b8-f615ef746f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:59.195144 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e159c-ef95-48a1-b8b8-f615ef746f77 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:59.205583 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1380/2729] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:21:59.209509 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4526c5e-66bb-4305-a530-b72d4d0318a5 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:59.220920 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4526c5e-66bb-4305-a530-b72d4d0318a5 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:59.226455 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510cf28-5247-41d6-9c9f-26d3ce247fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:59.227147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510cf28-5247-41d6-9c9f-26d3ce247fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:59.227532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510cf28-5247-41d6-9c9f-26d3ce247fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:59.227815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510cf28-5247-41d6-9c9f-26d3ce247fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:59.228556 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510cf28-5247-41d6-9c9f-26d3ce247fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:59.230198 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510cf28-5247-41d6-9c9f-26d3ce247fa3 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:59.231548 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:59.243223 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1350/2730] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:21:59.247256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e159c-ef95-48a1-b8b8-f615ef746f77 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:59.257349 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e159c-ef95-48a1-b8b8-f615ef746f77 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:59.262989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd50dd-97f4-4e58-a928-e1ad5ef5999b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:59.263904 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd50dd-97f4-4e58-a928-e1ad5ef5999b admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:59.264190 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd50dd-97f4-4e58-a928-e1ad5ef5999b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:59.264579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd50dd-97f4-4e58-a928-e1ad5ef5999b admin admin] PATH_INFO: `/v3/projects/863290e696b6400cb8d47ecd79ef32ad/users/d1495f4766bf43ba847b55507e1010c7/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:59.279762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd50dd-97f4-4e58-a928-e1ad5ef5999b admin admin] RBAC: Authorizing `identity:create_grant(project_id=863290e696b6400cb8d47ecd79ef32ad, user_id=d1495f4766bf43ba847b55507e1010c7, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:59.282537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd50dd-97f4-4e58-a928-e1ad5ef5999b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:59.298828 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1381/2731] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:21:59 2022] PUT /identity/v3/projects/863290e696b6400cb8d47ecd79ef32ad/users/d1495f4766bf43ba847b55507e1010c7/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:21:59.314432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62296e4a-10fd-4fe2-ae7e-c17bb3e8a7ff None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:59.314704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62296e4a-10fd-4fe2-ae7e-c17bb3e8a7ff None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:59.314992 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62296e4a-10fd-4fe2-ae7e-c17bb3e8a7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:59.345295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-62296e4a-10fd-4fe2-ae7e-c17bb3e8a7ff None None] MFA Rules not processed for user `d1495f4766bf43ba847b55507e1010c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:59.346580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-62296e4a-10fd-4fe2-ae7e-c17bb3e8a7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:59.412505 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1351/2732] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:21:59.424098 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd50dd-97f4-4e58-a928-e1ad5ef5999b admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:59.426124 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-71cd50dd-97f4-4e58-a928-e1ad5ef5999b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:21:59.504179 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd50dd-97f4-4e58-a928-e1ad5ef5999b admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:59.511260 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de813e9-9fe2-4ca7-933f-dfb4774f9664 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:59.511906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de813e9-9fe2-4ca7-933f-dfb4774f9664 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:59.512189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de813e9-9fe2-4ca7-933f-dfb4774f9664 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:59.512462 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de813e9-9fe2-4ca7-933f-dfb4774f9664 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:59.520457 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de813e9-9fe2-4ca7-933f-dfb4774f9664 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:59.522066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de813e9-9fe2-4ca7-933f-dfb4774f9664 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:59.530930 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1382/2733] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:21:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:21:59.747810 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62296e4a-10fd-4fe2-ae7e-c17bb3e8a7ff None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:21:59.820142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62296e4a-10fd-4fe2-ae7e-c17bb3e8a7ff None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:21:59.825368 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:21:59.826254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:59.826579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:59.826854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] PATH_INFO: `/v3/users/d1495f4766bf43ba847b55507e1010c7` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:59.828082 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1495f4766bf43ba847b55507e1010c7)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:21:59.830052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:21:59.887887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1495f4766bf43ba847b55507e1010c7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:59.888552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1495f4766bf43ba847b55507e1010c7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:59.889228 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1495f4766bf43ba847b55507e1010c7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:59.896343 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1495f4766bf43ba847b55507e1010c7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:59.908529 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1495f4766bf43ba847b55507e1010c7'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:21:59.923406 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1352/2734] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:21:59 2022] DELETE /identity/v3/users/d1495f4766bf43ba847b55507e1010c7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:21:59.928158 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c70af-de19-4155-b84d-577bc7169adb None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:21:59.928444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c70af-de19-4155-b84d-577bc7169adb None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:21:59.928720 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c70af-de19-4155-b84d-577bc7169adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:21:59.945885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-c09c70af-de19-4155-b84d-577bc7169adb None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:21:59.947104 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c09c70af-de19-4155-b84d-577bc7169adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:00.007980 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1383/2735] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:22:00.017932 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:00.019519 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:00.026418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:00.078450 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d986fe-f4b7-4eb8-a8aa-43c6c2a52e21 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:00.083174 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8438f1-7abc-43ca-88a4-91eb1b1ad358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:00.083881 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8438f1-7abc-43ca-88a4-91eb1b1ad358 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:00.084171 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8438f1-7abc-43ca-88a4-91eb1b1ad358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:00.084472 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8438f1-7abc-43ca-88a4-91eb1b1ad358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:00.095656 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8438f1-7abc-43ca-88a4-91eb1b1ad358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:00.097108 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8438f1-7abc-43ca-88a4-91eb1b1ad358 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:00.107525 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1353/2736] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:22:00.419232 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c70af-de19-4155-b84d-577bc7169adb None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:00.421391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c09c70af-de19-4155-b84d-577bc7169adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:00.434071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c70af-de19-4155-b84d-577bc7169adb None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:00.438703 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:00.439593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:00.439872 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:00.440144 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] PATH_INFO: `/v3/projects/863290e696b6400cb8d47ecd79ef32ad` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:00.441373 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=863290e696b6400cb8d47ecd79ef32ad)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:00.443817 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:00.487927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:INSERT=2,keystone:SELECT=95,keystone:DELETE=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:00.496170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '863290e696b6400cb8d47ecd79ef32ad', 'request_id': 'req-c73c0981-a296-4e2b-99da-ce1ca570fc3b'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:00.504471 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '863290e696b6400cb8d47ecd79ef32ad', 'request_id': 'req-c73c0981-a296-4e2b-99da-ce1ca570fc3b'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:00.513741 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '863290e696b6400cb8d47ecd79ef32ad', 'request_id': 'req-c73c0981-a296-4e2b-99da-ce1ca570fc3b'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:00.529652 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] The token cache is being invalidate because project 863290e696b6400cb8d47ecd79ef32ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:22:00.529945 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:00.530694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c73c0981-a296-4e2b-99da-ce1ca570fc3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:00.532379 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1384/2737] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:22:00 2022] DELETE /identity/v3/projects/863290e696b6400cb8d47ecd79ef32ad => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:22:00.537610 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e5b92-6a18-4633-9a63-3faeaa57e92c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:00.537903 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e5b92-6a18-4633-9a63-3faeaa57e92c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:00.538182 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e5b92-6a18-4633-9a63-3faeaa57e92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:00.559098 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-135e5b92-6a18-4633-9a63-3faeaa57e92c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:22:00.560468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-135e5b92-6a18-4633-9a63-3faeaa57e92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:00.623161 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1354/2738] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:22:00.631303 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa3dd3-ffe0-47b7-8d0b-c762438eb727 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:00.631597 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa3dd3-ffe0-47b7-8d0b-c762438eb727 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:00.631880 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa3dd3-ffe0-47b7-8d0b-c762438eb727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:00.650377 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-e1fa3dd3-ffe0-47b7-8d0b-c762438eb727 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:22:00.651593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-e1fa3dd3-ffe0-47b7-8d0b-c762438eb727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:00.660337 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1385/2739] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:22:00.664701 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e5b92-6a18-4633-9a63-3faeaa57e92c None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:00.675195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e5b92-6a18-4633-9a63-3faeaa57e92c None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:00.680725 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f36e1c-63f0-4ccc-bfba-48af2e251d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:00.681383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f36e1c-63f0-4ccc-bfba-48af2e251d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:00.681662 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f36e1c-63f0-4ccc-bfba-48af2e251d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:00.681933 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f36e1c-63f0-4ccc-bfba-48af2e251d4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:00.682697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f36e1c-63f0-4ccc-bfba-48af2e251d4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:22:00.683016 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f36e1c-63f0-4ccc-bfba-48af2e251d4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:00.684620 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f36e1c-63f0-4ccc-bfba-48af2e251d4a admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:00.697404 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1355/2740] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:22:00.701737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa3dd3-ffe0-47b7-8d0b-c762438eb727 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:00.711773 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa3dd3-ffe0-47b7-8d0b-c762438eb727 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:00.717050 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21280eb4-309a-4dff-b2ef-5a46c3ea3f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:00.717718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280eb4-309a-4dff-b2ef-5a46c3ea3f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:00.718007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280eb4-309a-4dff-b2ef-5a46c3ea3f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:00.718274 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280eb4-309a-4dff-b2ef-5a46c3ea3f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:00.718989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21280eb4-309a-4dff-b2ef-5a46c3ea3f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:00.721002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21280eb4-309a-4dff-b2ef-5a46c3ea3f1c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:00.748362 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1386/2741] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:22:00 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:22:00.753377 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f36e1c-63f0-4ccc-bfba-48af2e251d4a admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:00.764837 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f36e1c-63f0-4ccc-bfba-48af2e251d4a admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:00.832798 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264164e5-18e5-40f7-9979-7ab86f954d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:00.833482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264164e5-18e5-40f7-9979-7ab86f954d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:00.833770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264164e5-18e5-40f7-9979-7ab86f954d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:00.834036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264164e5-18e5-40f7-9979-7ab86f954d0e admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:00.834775 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264164e5-18e5-40f7-9979-7ab86f954d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:00.837066 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264164e5-18e5-40f7-9979-7ab86f954d0e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:00.858287 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1356/2742] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:22:00 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:22:00.862948 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21280eb4-309a-4dff-b2ef-5a46c3ea3f1c admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:00.874053 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21280eb4-309a-4dff-b2ef-5a46c3ea3f1c admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:00.879521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4d753-1fa7-4207-8106-62d1aa01a912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:00.880197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4d753-1fa7-4207-8106-62d1aa01a912 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:00.880487 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4d753-1fa7-4207-8106-62d1aa01a912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:00.880766 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4d753-1fa7-4207-8106-62d1aa01a912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:00.881515 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4d753-1fa7-4207-8106-62d1aa01a912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:00.883212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4d753-1fa7-4207-8106-62d1aa01a912 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:00.894037 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1387/2743] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:22:00.898033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264164e5-18e5-40f7-9979-7ab86f954d0e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:00.909581 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264164e5-18e5-40f7-9979-7ab86f954d0e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:00.915131 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a94ee-b8c1-4cc2-bd40-4bb3b6684525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:00.915821 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a94ee-b8c1-4cc2-bd40-4bb3b6684525 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:00.916091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a94ee-b8c1-4cc2-bd40-4bb3b6684525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:00.916376 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a94ee-b8c1-4cc2-bd40-4bb3b6684525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:00.917079 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a94ee-b8c1-4cc2-bd40-4bb3b6684525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:00.918761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a94ee-b8c1-4cc2-bd40-4bb3b6684525 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:00.929498 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1357/2744] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:22:00.933726 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4d753-1fa7-4207-8106-62d1aa01a912 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:00.944123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4d753-1fa7-4207-8106-62d1aa01a912 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:00.949379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7023c-f80e-49fc-a0af-56af89bdf8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:00.950326 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7023c-f80e-49fc-a0af-56af89bdf8b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:00.950614 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7023c-f80e-49fc-a0af-56af89bdf8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:00.950882 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7023c-f80e-49fc-a0af-56af89bdf8b7 admin admin] PATH_INFO: `/v3/projects/d4adbaed00064c1386e851f6d0a1ea16/users/3f43ad01c29a4e6a8152a67dd0b2ee97/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:00.966458 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7023c-f80e-49fc-a0af-56af89bdf8b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4adbaed00064c1386e851f6d0a1ea16, user_id=3f43ad01c29a4e6a8152a67dd0b2ee97, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:00.969128 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7023c-f80e-49fc-a0af-56af89bdf8b7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:00.984223 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1388/2745] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:22:00 2022] PUT /identity/v3/projects/d4adbaed00064c1386e851f6d0a1ea16/users/3f43ad01c29a4e6a8152a67dd0b2ee97/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:22:00.994809 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386f3e4-50eb-4d91-89db-516683fa2e53 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:00.995096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386f3e4-50eb-4d91-89db-516683fa2e53 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:00.995370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386f3e4-50eb-4d91-89db-516683fa2e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:01.020506 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:01.023586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-c386f3e4-50eb-4d91-89db-516683fa2e53 None None] MFA Rules not processed for user `3f43ad01c29a4e6a8152a67dd0b2ee97`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:22:01.025523 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-c386f3e4-50eb-4d91-89db-516683fa2e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:01.092696 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1358/2746] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:22:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:22:01.106905 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7023c-f80e-49fc-a0af-56af89bdf8b7 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:01.108893 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4be7023c-f80e-49fc-a0af-56af89bdf8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:01.174475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7023c-f80e-49fc-a0af-56af89bdf8b7 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:01.178428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e128de-8ffd-48ad-9891-2aab13d02e22 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b97736e49e2a4bfb9cf7093cfa8b2b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:01.179113 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e128de-8ffd-48ad-9891-2aab13d02e22 service glance] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:01.179400 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e128de-8ffd-48ad-9891-2aab13d02e22 service glance] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:01.179680 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e128de-8ffd-48ad-9891-2aab13d02e22 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:01.188707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e128de-8ffd-48ad-9891-2aab13d02e22 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:01.190154 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e128de-8ffd-48ad-9891-2aab13d02e22 service glance] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:01.199452 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1389/2747] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:22:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:22:11.089329 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=26 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:11.195300 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:19.929997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386f3e4-50eb-4d91-89db-516683fa2e53 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:19.995908 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386f3e4-50eb-4d91-89db-516683fa2e53 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:20.000914 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:20.001782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:20.002070 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:20.002369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] PATH_INFO: `/v3/users/3f43ad01c29a4e6a8152a67dd0b2ee97` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:20.003647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f43ad01c29a4e6a8152a67dd0b2ee97)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:20.005592 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:20.054880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f43ad01c29a4e6a8152a67dd0b2ee97'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:20.055510 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f43ad01c29a4e6a8152a67dd0b2ee97'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:20.056129 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f43ad01c29a4e6a8152a67dd0b2ee97'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:20.062924 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f43ad01c29a4e6a8152a67dd0b2ee97'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:20.074781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f43ad01c29a4e6a8152a67dd0b2ee97'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:20.090365 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1359/2748] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:22:19 2022] DELETE /identity/v3/users/3f43ad01c29a4e6a8152a67dd0b2ee97 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:22:20.095159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac866d-e393-4176-9cd6-b406bad4d4ae None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:20.095669 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac866d-e393-4176-9cd6-b406bad4d4ae None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:20.095793 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac866d-e393-4176-9cd6-b406bad4d4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:20.113629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-afac866d-e393-4176-9cd6-b406bad4d4ae None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:22:20.114864 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-afac866d-e393-4176-9cd6-b406bad4d4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:20.176603 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1390/2749] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:22:20.186434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:20.188089 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:20.251832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb30a1-c02b-472d-b342-e684393c24ae admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:20.257058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca1832-4043-418f-bb28-2564c277ca09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:20.257683 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca1832-4043-418f-bb28-2564c277ca09 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:20.257989 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca1832-4043-418f-bb28-2564c277ca09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:20.258283 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca1832-4043-418f-bb28-2564c277ca09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:20.268539 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca1832-4043-418f-bb28-2564c277ca09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:20.269971 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca1832-4043-418f-bb28-2564c277ca09 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:20.280132 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1360/2750] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:22:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:22:20.571015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac866d-e393-4176-9cd6-b406bad4d4ae None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:20.572664 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-afac866d-e393-4176-9cd6-b406bad4d4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:20.586070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac866d-e393-4176-9cd6-b406bad4d4ae None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:20.592189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:20.593024 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:20.593299 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:20.593569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] PATH_INFO: `/v3/projects/d4adbaed00064c1386e851f6d0a1ea16` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:20.595776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4adbaed00064c1386e851f6d0a1ea16)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:20.597708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:20.649263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4adbaed00064c1386e851f6d0a1ea16', 'request_id': 'req-838aed06-3447-45da-8ee3-c4a3ef4977b6'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:20.654696 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4adbaed00064c1386e851f6d0a1ea16', 'request_id': 'req-838aed06-3447-45da-8ee3-c4a3ef4977b6'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:20.662708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4adbaed00064c1386e851f6d0a1ea16', 'request_id': 'req-838aed06-3447-45da-8ee3-c4a3ef4977b6'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:20.678856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] The token cache is being invalidate because project d4adbaed00064c1386e851f6d0a1ea16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:22:20.679148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:20.679870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-838aed06-3447-45da-8ee3-c4a3ef4977b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:22:20.685034 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1391/2751] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:22:20 2022] DELETE /identity/v3/projects/d4adbaed00064c1386e851f6d0a1ea16 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:22:20.686694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b40dc5-8369-4fb3-8877-56878dfd8f58 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:20.686986 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b40dc5-8369-4fb3-8877-56878dfd8f58 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:20.687360 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b40dc5-8369-4fb3-8877-56878dfd8f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:20.706949 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-49b40dc5-8369-4fb3-8877-56878dfd8f58 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:22:20.708169 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-49b40dc5-8369-4fb3-8877-56878dfd8f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:20.725520 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:20.774577 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1361/2752] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:22:20.783940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dac232-149f-4443-8418-377e0fea9171 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:20.784229 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dac232-149f-4443-8418-377e0fea9171 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:20.784504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dac232-149f-4443-8418-377e0fea9171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:20.803629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-d5dac232-149f-4443-8418-377e0fea9171 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:22:20.804832 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d5dac232-149f-4443-8418-377e0fea9171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:20.813796 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1392/2753] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:22:20.818164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b40dc5-8369-4fb3-8877-56878dfd8f58 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:20.828084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b40dc5-8369-4fb3-8877-56878dfd8f58 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:20.834153 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299600d3-abc7-4d78-bd7a-252d4d3c7801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:20.834838 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299600d3-abc7-4d78-bd7a-252d4d3c7801 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:20.835120 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299600d3-abc7-4d78-bd7a-252d4d3c7801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:20.835396 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299600d3-abc7-4d78-bd7a-252d4d3c7801 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:20.836435 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299600d3-abc7-4d78-bd7a-252d4d3c7801 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:22:20.836741 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299600d3-abc7-4d78-bd7a-252d4d3c7801 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:20.838401 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299600d3-abc7-4d78-bd7a-252d4d3c7801 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:20.850113 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1362/2754] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:22:20.854475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dac232-149f-4443-8418-377e0fea9171 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:20.863913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dac232-149f-4443-8418-377e0fea9171 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:20.869870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81e1b2-2cb7-4732-bcda-bafa7a1b50cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:20.870564 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81e1b2-2cb7-4732-bcda-bafa7a1b50cd admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:20.870848 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81e1b2-2cb7-4732-bcda-bafa7a1b50cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:20.871117 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81e1b2-2cb7-4732-bcda-bafa7a1b50cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:20.871799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81e1b2-2cb7-4732-bcda-bafa7a1b50cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:20.873700 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81e1b2-2cb7-4732-bcda-bafa7a1b50cd admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:20.902608 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1393/2755] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:22:20 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:22:20.906819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299600d3-abc7-4d78-bd7a-252d4d3c7801 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:20.917503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299600d3-abc7-4d78-bd7a-252d4d3c7801 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:20.978803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f39ba-3913-4ccd-a181-8dd4f1fcd1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:20.979496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f39ba-3913-4ccd-a181-8dd4f1fcd1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:20.979742 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f39ba-3913-4ccd-a181-8dd4f1fcd1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:20.980021 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f39ba-3913-4ccd-a181-8dd4f1fcd1e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:20.980714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f39ba-3913-4ccd-a181-8dd4f1fcd1e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:20.982672 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f39ba-3913-4ccd-a181-8dd4f1fcd1e1 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:21.002838 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1363/2756] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:22:20 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:22:21.007109 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81e1b2-2cb7-4732-bcda-bafa7a1b50cd admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:21.017247 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81e1b2-2cb7-4732-bcda-bafa7a1b50cd admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:21.021858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af779f-0557-4317-852a-71980db4df54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:21.022554 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af779f-0557-4317-852a-71980db4df54 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:21.022912 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af779f-0557-4317-852a-71980db4df54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:21.023219 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af779f-0557-4317-852a-71980db4df54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:21.023886 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af779f-0557-4317-852a-71980db4df54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:21.025928 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af779f-0557-4317-852a-71980db4df54 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:21.044087 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1394/2757] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:22:21.049818 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f39ba-3913-4ccd-a181-8dd4f1fcd1e1 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:21.060604 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f39ba-3913-4ccd-a181-8dd4f1fcd1e1 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:21.065863 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4b31c-122a-4a4f-9635-c3ff345db16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:21.066562 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4b31c-122a-4a4f-9635-c3ff345db16f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:21.066848 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4b31c-122a-4a4f-9635-c3ff345db16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:21.067130 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4b31c-122a-4a4f-9635-c3ff345db16f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:21.067866 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4b31c-122a-4a4f-9635-c3ff345db16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:21.069498 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4b31c-122a-4a4f-9635-c3ff345db16f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:21.079500 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1364/2758] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:22:21.083581 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af779f-0557-4317-852a-71980db4df54 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:21.094215 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af779f-0557-4317-852a-71980db4df54 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:21.100035 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3889264-4d1a-41dd-a600-da2ab98128f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:21.100949 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3889264-4d1a-41dd-a600-da2ab98128f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:21.101237 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3889264-4d1a-41dd-a600-da2ab98128f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:21.101497 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3889264-4d1a-41dd-a600-da2ab98128f2 admin admin] PATH_INFO: `/v3/projects/5ab94d9d4e204ffe9904139a9cf7c9ed/users/c2e5b1e9bab54b63af28a0c9eb295610/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:21.118444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3889264-4d1a-41dd-a600-da2ab98128f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab94d9d4e204ffe9904139a9cf7c9ed, user_id=c2e5b1e9bab54b63af28a0c9eb295610, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:21.121164 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3889264-4d1a-41dd-a600-da2ab98128f2 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:21.135308 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1395/2759] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:22:21 2022] PUT /identity/v3/projects/5ab94d9d4e204ffe9904139a9cf7c9ed/users/c2e5b1e9bab54b63af28a0c9eb295610/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:22:21.140545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209082c9-074c-4beb-b0be-c7405cdb7009 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:21.140839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209082c9-074c-4beb-b0be-c7405cdb7009 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:21.141118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209082c9-074c-4beb-b0be-c7405cdb7009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:21.158952 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-209082c9-074c-4beb-b0be-c7405cdb7009 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:22:21.160121 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-209082c9-074c-4beb-b0be-c7405cdb7009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:21.224922 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1365/2760] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:22:21.237866 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3889264-4d1a-41dd-a600-da2ab98128f2 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:21.239899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-f3889264-4d1a-41dd-a600-da2ab98128f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:21.263890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:21.300889 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3889264-4d1a-41dd-a600-da2ab98128f2 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:21.304931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc88a73-5094-41c1-a7d5-0153fa96d0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:21.305580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc88a73-5094-41c1-a7d5-0153fa96d0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:21.305867 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc88a73-5094-41c1-a7d5-0153fa96d0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:21.306140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc88a73-5094-41c1-a7d5-0153fa96d0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:21.315217 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc88a73-5094-41c1-a7d5-0153fa96d0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:21.316650 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc88a73-5094-41c1-a7d5-0153fa96d0d1 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:21.325769 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1396/2761] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:22:21.944367 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbae15-3d9a-4b5b-b042-8d0560fc6cf8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:21.944697 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbae15-3d9a-4b5b-b042-8d0560fc6cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:21.944977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbae15-3d9a-4b5b-b042-8d0560fc6cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:21.974997 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2efbae15-3d9a-4b5b-b042-8d0560fc6cf8 None None] MFA Rules not processed for user `c2e5b1e9bab54b63af28a0c9eb295610`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:22:21.976273 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2efbae15-3d9a-4b5b-b042-8d0560fc6cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:21.994489 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:22.047900 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1366/2762] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:22:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:22:22.058853 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc88a73-5094-41c1-a7d5-0153fa96d0d1 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:22.060394 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0cc88a73-5094-41c1-a7d5-0153fa96d0d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:22.119492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc88a73-5094-41c1-a7d5-0153fa96d0d1 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:22.123256 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8a23-aa18-4705-9235-edae1f2a68c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:22.123902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8a23-aa18-4705-9235-edae1f2a68c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:22.124180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8a23-aa18-4705-9235-edae1f2a68c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:22.124445 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8a23-aa18-4705-9235-edae1f2a68c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:22.133331 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8a23-aa18-4705-9235-edae1f2a68c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:22.134855 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8a23-aa18-4705-9235-edae1f2a68c0 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:22.144094 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1397/2763] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:22:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:22:23.502485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4b4e-1d2b-41dc-953a-9543cdd2439b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:23.503031 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4b4e-1d2b-41dc-953a-9543cdd2439b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:23.503291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4b4e-1d2b-41dc-953a-9543cdd2439b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:23.523348 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-844d4b4e-1d2b-41dc-953a-9543cdd2439b None None] MFA Rules not processed for user `c2e5b1e9bab54b63af28a0c9eb295610`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:22:23.524581 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-844d4b4e-1d2b-41dc-953a-9543cdd2439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:23.532077 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1367/2764] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:22:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:22:25.204086 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8a23-aa18-4705-9235-edae1f2a68c0 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:25.215266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8a23-aa18-4705-9235-edae1f2a68c0 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:25.221761 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa46357-a4a9-4ac7-9105-c441aca22004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:25.222599 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa46357-a4a9-4ac7-9105-c441aca22004 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:25.223020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa46357-a4a9-4ac7-9105-c441aca22004 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:25.223414 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa46357-a4a9-4ac7-9105-c441aca22004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:25.226114 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6fa46357-a4a9-4ac7-9105-c441aca22004 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:25.311097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa46357-a4a9-4ac7-9105-c441aca22004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:25.312618 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa46357-a4a9-4ac7-9105-c441aca22004 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:25.324197 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1398/2765] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:22:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:22:26.434013 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4b4e-1d2b-41dc-953a-9543cdd2439b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:26.444638 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4b4e-1d2b-41dc-953a-9543cdd2439b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:26.448542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc1989-2129-4ed0-ac50-759bbd0764ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:26.449226 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc1989-2129-4ed0-ac50-759bbd0764ba service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:26.449492 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc1989-2129-4ed0-ac50-759bbd0764ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:26.449781 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc1989-2129-4ed0-ac50-759bbd0764ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:26.451887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d7fc1989-2129-4ed0-ac50-759bbd0764ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:26.466758 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc1989-2129-4ed0-ac50-759bbd0764ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:26.468582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc1989-2129-4ed0-ac50-759bbd0764ba service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:26.480848 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1368/2766] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:22:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:22:35.319110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=77 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:36.475820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=28 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:38.926750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf27c9f-3afc-4676-89ce-cc23c94bf805 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:38.927270 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf27c9f-3afc-4676-89ce-cc23c94bf805 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:38.927400 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf27c9f-3afc-4676-89ce-cc23c94bf805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:38.945523 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-daf27c9f-3afc-4676-89ce-cc23c94bf805 None None] MFA Rules not processed for user `c2e5b1e9bab54b63af28a0c9eb295610`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:22:38.946911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-daf27c9f-3afc-4676-89ce-cc23c94bf805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:38.954803 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1399/2767] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:22:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:22:48.940694 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:57.064777 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553d73-65ed-4396-b3f7-16c8d4e4bb48 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:57.065310 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553d73-65ed-4396-b3f7-16c8d4e4bb48 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:57.065310 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553d73-65ed-4396-b3f7-16c8d4e4bb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:57.083230 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-12553d73-65ed-4396-b3f7-16c8d4e4bb48 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:22:57.084426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-12553d73-65ed-4396-b3f7-16c8d4e4bb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:57.148366 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1369/2768] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:22:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:22:57.196884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf27c9f-3afc-4676-89ce-cc23c94bf805 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:22:57.198573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-daf27c9f-3afc-4676-89ce-cc23c94bf805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:22:57.279933 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf27c9f-3afc-4676-89ce-cc23c94bf805 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:22:57.285767 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73433bf-0621-4f19-bbc4-28ed39c95d3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:22:57.286387 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73433bf-0621-4f19-bbc4-28ed39c95d3d service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:22:57.286672 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73433bf-0621-4f19-bbc4-28ed39c95d3d service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:22:57.286947 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73433bf-0621-4f19-bbc4-28ed39c95d3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:22:57.295520 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73433bf-0621-4f19-bbc4-28ed39c95d3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:22:57.297111 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73433bf-0621-4f19-bbc4-28ed39c95d3d service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:22:57.305218 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1400/2769] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:22:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:23:07.141262 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=25 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:07.302118 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=32 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:11.669712 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f433270-b863-434f-9bdc-854da58543c1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:11.670267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f433270-b863-434f-9bdc-854da58543c1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:11.670267 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f433270-b863-434f-9bdc-854da58543c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:11.689988 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-5f433270-b863-434f-9bdc-854da58543c1 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:23:11.691241 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5f433270-b863-434f-9bdc-854da58543c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:11.699036 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1370/2770] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jul 5 10:23:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:23:11.711591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73433bf-0621-4f19-bbc4-28ed39c95d3d service swift] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:11.722783 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73433bf-0621-4f19-bbc4-28ed39c95d3d service swift] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:11.729752 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423f5f0-fdd3-4ee4-b608-d47599585935 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:11.730421 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423f5f0-fdd3-4ee4-b608-d47599585935 service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:11.730701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423f5f0-fdd3-4ee4-b608-d47599585935 service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:11.730970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423f5f0-fdd3-4ee4-b608-d47599585935 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:11.739896 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423f5f0-fdd3-4ee4-b608-d47599585935 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:11.741571 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423f5f0-fdd3-4ee4-b608-d47599585935 service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:11.750116 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1401/2771] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:23:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:23:13.643237 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f433270-b863-434f-9bdc-854da58543c1 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:13.644347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5f433270-b863-434f-9bdc-854da58543c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:13.700575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423f5f0-fdd3-4ee4-b608-d47599585935 service swift] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:13.713206 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423f5f0-fdd3-4ee4-b608-d47599585935 service swift] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:13.718875 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca667af1-c8e2-4bd9-bdec-1e59137e684a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:13.719777 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca667af1-c8e2-4bd9-bdec-1e59137e684a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:13.720065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca667af1-c8e2-4bd9-bdec-1e59137e684a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:13.720349 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca667af1-c8e2-4bd9-bdec-1e59137e684a admin admin] PATH_INFO: `/v3/users/c2e5b1e9bab54b63af28a0c9eb295610` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:13.721644 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca667af1-c8e2-4bd9-bdec-1e59137e684a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e5b1e9bab54b63af28a0c9eb295610)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:13.723648 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca667af1-c8e2-4bd9-bdec-1e59137e684a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:13.727796 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f433270-b863-434f-9bdc-854da58543c1 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:13.734413 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0235ea-a672-4473-8f39-2a5efec3fc20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:13.735110 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0235ea-a672-4473-8f39-2a5efec3fc20 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:13.735398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0235ea-a672-4473-8f39-2a5efec3fc20 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:13.735678 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0235ea-a672-4473-8f39-2a5efec3fc20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:13.737512 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5e0235ea-a672-4473-8f39-2a5efec3fc20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:13.751422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0235ea-a672-4473-8f39-2a5efec3fc20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:13.757295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0235ea-a672-4473-8f39-2a5efec3fc20 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:13.770492 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1371/2772] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:23:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:23:13.775070 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ca667af1-c8e2-4bd9-bdec-1e59137e684a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e5b1e9bab54b63af28a0c9eb295610'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:13.775834 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ca667af1-c8e2-4bd9-bdec-1e59137e684a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e5b1e9bab54b63af28a0c9eb295610'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:13.788039 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ca667af1-c8e2-4bd9-bdec-1e59137e684a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e5b1e9bab54b63af28a0c9eb295610'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:13.794984 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ca667af1-c8e2-4bd9-bdec-1e59137e684a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e5b1e9bab54b63af28a0c9eb295610'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:13.795676 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ca667af1-c8e2-4bd9-bdec-1e59137e684a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e5b1e9bab54b63af28a0c9eb295610'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:13.810037 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1402/2773] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:23:13 2022] DELETE /identity/v3/users/c2e5b1e9bab54b63af28a0c9eb295610 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:23:13.998337 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0235ea-a672-4473-8f39-2a5efec3fc20 service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:14.011906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0235ea-a672-4473-8f39-2a5efec3fc20 service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:14.071270 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:14.072251 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:14.072536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:14.072815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] PATH_INFO: `/v3/projects/5ab94d9d4e204ffe9904139a9cf7c9ed` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:14.074002 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab94d9d4e204ffe9904139a9cf7c9ed)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:14.075954 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:14.122874 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab94d9d4e204ffe9904139a9cf7c9ed', 'request_id': 'req-7eca867f-e210-4a2a-8220-1db3820304bb'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:14.131475 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab94d9d4e204ffe9904139a9cf7c9ed', 'request_id': 'req-7eca867f-e210-4a2a-8220-1db3820304bb'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:14.137501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab94d9d4e204ffe9904139a9cf7c9ed', 'request_id': 'req-7eca867f-e210-4a2a-8220-1db3820304bb'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:14.148912 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] The token cache is being invalidate because project 5ab94d9d4e204ffe9904139a9cf7c9ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:23:14.149201 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:14.149906 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-7eca867f-e210-4a2a-8220-1db3820304bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:14.151474 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1372/2774] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:23:13 2022] DELETE /identity/v3/projects/5ab94d9d4e204ffe9904139a9cf7c9ed => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:23:14.156635 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b66b91-e8e2-47e1-8f24-d5df3ea57868 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:14.156933 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b66b91-e8e2-47e1-8f24-d5df3ea57868 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:14.157214 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b66b91-e8e2-47e1-8f24-d5df3ea57868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:14.176899 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-19b66b91-e8e2-47e1-8f24-d5df3ea57868 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:23:14.178187 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-19b66b91-e8e2-47e1-8f24-d5df3ea57868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:14.242409 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1403/2775] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:23:14.250274 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba86001-7d55-4d64-9199-bea4175a32a0 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:14.250566 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba86001-7d55-4d64-9199-bea4175a32a0 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:14.250845 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba86001-7d55-4d64-9199-bea4175a32a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:14.272364 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-fba86001-7d55-4d64-9199-bea4175a32a0 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:23:14.273589 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-fba86001-7d55-4d64-9199-bea4175a32a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:14.282592 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1373/2776] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:23:14.287633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b66b91-e8e2-47e1-8f24-d5df3ea57868 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:14.302544 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b66b91-e8e2-47e1-8f24-d5df3ea57868 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:14.309376 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296c6b3-3b39-42e2-b30c-2b8037472819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:14.310043 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296c6b3-3b39-42e2-b30c-2b8037472819 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:14.310365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296c6b3-3b39-42e2-b30c-2b8037472819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:14.310643 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296c6b3-3b39-42e2-b30c-2b8037472819 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:14.311371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296c6b3-3b39-42e2-b30c-2b8037472819 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:23:14.311678 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296c6b3-3b39-42e2-b30c-2b8037472819 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:14.313289 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296c6b3-3b39-42e2-b30c-2b8037472819 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:14.325376 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1404/2777] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:23:14.329823 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba86001-7d55-4d64-9199-bea4175a32a0 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:14.339778 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba86001-7d55-4d64-9199-bea4175a32a0 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:14.345731 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a737ff-a967-4e40-a341-9b25216ab014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:14.346426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a737ff-a967-4e40-a341-9b25216ab014 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:14.346708 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a737ff-a967-4e40-a341-9b25216ab014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:14.346988 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a737ff-a967-4e40-a341-9b25216ab014 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:14.347690 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a737ff-a967-4e40-a341-9b25216ab014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:14.349605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a737ff-a967-4e40-a341-9b25216ab014 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:14.353024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:14.383437 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1374/2778] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:23:14 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:23:14.387575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296c6b3-3b39-42e2-b30c-2b8037472819 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:14.398503 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296c6b3-3b39-42e2-b30c-2b8037472819 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:14.442829 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:14.457530 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560569e7-688f-43fe-b582-e435638ec807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:14.458307 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560569e7-688f-43fe-b582-e435638ec807 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:14.458596 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560569e7-688f-43fe-b582-e435638ec807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:14.458879 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560569e7-688f-43fe-b582-e435638ec807 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:14.459922 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560569e7-688f-43fe-b582-e435638ec807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:14.461901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560569e7-688f-43fe-b582-e435638ec807 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:14.484334 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1405/2779] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:23:14 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:23:14.488407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a737ff-a967-4e40-a341-9b25216ab014 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:14.499405 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a737ff-a967-4e40-a341-9b25216ab014 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:14.505831 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72f5be-536d-4d85-bf0c-6350766cd2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:14.506586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72f5be-536d-4d85-bf0c-6350766cd2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:14.506861 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72f5be-536d-4d85-bf0c-6350766cd2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:14.507134 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72f5be-536d-4d85-bf0c-6350766cd2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:14.507864 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72f5be-536d-4d85-bf0c-6350766cd2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:14.509830 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72f5be-536d-4d85-bf0c-6350766cd2a7 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:14.520443 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1375/2780] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:23:14.524629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560569e7-688f-43fe-b582-e435638ec807 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:14.534988 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560569e7-688f-43fe-b582-e435638ec807 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:14.539884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ab451-fd1e-4deb-958d-c795e642c446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:14.540540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ab451-fd1e-4deb-958d-c795e642c446 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:14.540832 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ab451-fd1e-4deb-958d-c795e642c446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:14.541100 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ab451-fd1e-4deb-958d-c795e642c446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:14.541819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ab451-fd1e-4deb-958d-c795e642c446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:14.543448 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ab451-fd1e-4deb-958d-c795e642c446 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:14.553298 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1406/2781] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:23:14.559969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72f5be-536d-4d85-bf0c-6350766cd2a7 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:14.569488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72f5be-536d-4d85-bf0c-6350766cd2a7 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:14.575456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46f66e-7d13-471b-826f-09a18b885f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:14.576392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46f66e-7d13-471b-826f-09a18b885f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:14.576679 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46f66e-7d13-471b-826f-09a18b885f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:14.576990 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46f66e-7d13-471b-826f-09a18b885f6e admin admin] PATH_INFO: `/v3/projects/cbfc99c4dc004ee69e0f6ccbf83917d9/users/4f53d2279833435b9114128cd020bdc9/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:14.595385 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46f66e-7d13-471b-826f-09a18b885f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfc99c4dc004ee69e0f6ccbf83917d9, user_id=4f53d2279833435b9114128cd020bdc9, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:14.598027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46f66e-7d13-471b-826f-09a18b885f6e admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:14.615847 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1376/2782] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:23:14 2022] PUT /identity/v3/projects/cbfc99c4dc004ee69e0f6ccbf83917d9/users/4f53d2279833435b9114128cd020bdc9/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:23:14.682989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4292c8-566d-41fe-ba52-555d72ee261b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:14.683249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4292c8-566d-41fe-ba52-555d72ee261b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:14.683540 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4292c8-566d-41fe-ba52-555d72ee261b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:14.711252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-eb4292c8-566d-41fe-ba52-555d72ee261b None None] MFA Rules not processed for user `4f53d2279833435b9114128cd020bdc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:23:14.712434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-eb4292c8-566d-41fe-ba52-555d72ee261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:14.779998 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1407/2783] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:23:14 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:23:14.791612 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46f66e-7d13-471b-826f-09a18b885f6e admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:14.793405 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9f46f66e-7d13-471b-826f-09a18b885f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:14.862024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46f66e-7d13-471b-826f-09a18b885f6e admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:14.866596 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4fb07-8c61-4848-8ec4-48239ad16f66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:14.867254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4fb07-8c61-4848-8ec4-48239ad16f66 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:14.867537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4fb07-8c61-4848-8ec4-48239ad16f66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:14.867809 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4fb07-8c61-4848-8ec4-48239ad16f66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:14.876714 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4fb07-8c61-4848-8ec4-48239ad16f66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:14.878171 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4fb07-8c61-4848-8ec4-48239ad16f66 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:14.887702 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1377/2784] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:23:14 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:23:24.777097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:24.883316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:31.204059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4292c8-566d-41fe-ba52-555d72ee261b None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:31.206106 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-eb4292c8-566d-41fe-ba52-555d72ee261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:31.267031 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4292c8-566d-41fe-ba52-555d72ee261b None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:31.271603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e9066-dd72-42ae-9bcd-cb3e2e1f92b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:31.272257 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e9066-dd72-42ae-9bcd-cb3e2e1f92b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:31.272543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e9066-dd72-42ae-9bcd-cb3e2e1f92b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:31.272858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e9066-dd72-42ae-9bcd-cb3e2e1f92b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:31.275434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0c4e9066-dd72-42ae-9bcd-cb3e2e1f92b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:31.288636 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e9066-dd72-42ae-9bcd-cb3e2e1f92b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:31.290149 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e9066-dd72-42ae-9bcd-cb3e2e1f92b8 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:31.298986 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1408/2785] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:23:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:23:41.294998 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=26 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:50.140702 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4fb07-8c61-4848-8ec4-48239ad16f66 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:50.211461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4fb07-8c61-4848-8ec4-48239ad16f66 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:50.217068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:50.218066 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:50.218389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:50.218675 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] PATH_INFO: `/v3/users/4f53d2279833435b9114128cd020bdc9` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:50.220033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f53d2279833435b9114128cd020bdc9)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:50.221995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:50.275582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f53d2279833435b9114128cd020bdc9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:50.276483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f53d2279833435b9114128cd020bdc9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:50.277372 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f53d2279833435b9114128cd020bdc9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:50.288461 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f53d2279833435b9114128cd020bdc9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:50.302160 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f53d2279833435b9114128cd020bdc9'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:50.318359 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1378/2786] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:23:50 2022] DELETE /identity/v3/users/4f53d2279833435b9114128cd020bdc9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:23:50.324281 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad859e5-932d-4dad-af3b-c17f7cfc3393 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:50.324800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad859e5-932d-4dad-af3b-c17f7cfc3393 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:50.324901 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad859e5-932d-4dad-af3b-c17f7cfc3393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:50.344711 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-2ad859e5-932d-4dad-af3b-c17f7cfc3393 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:23:50.345933 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2ad859e5-932d-4dad-af3b-c17f7cfc3393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:50.405337 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1409/2787] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:23:50.415254 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:50.416801 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:50.475018 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5f337-3483-4bd9-8889-6515ce05ad50 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:50.479436 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ca4d8-c2ac-43e1-8aae-6efccb56a28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:50.480081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca4d8-c2ac-43e1-8aae-6efccb56a28e service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:50.480370 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca4d8-c2ac-43e1-8aae-6efccb56a28e service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:50.480646 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca4d8-c2ac-43e1-8aae-6efccb56a28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:50.490260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ca4d8-c2ac-43e1-8aae-6efccb56a28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:50.491752 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ca4d8-c2ac-43e1-8aae-6efccb56a28e service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:50.501305 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1379/2788] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:23:50.780881 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad859e5-932d-4dad-af3b-c17f7cfc3393 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:23:50.782521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-2ad859e5-932d-4dad-af3b-c17f7cfc3393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:23:50.794838 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad859e5-932d-4dad-af3b-c17f7cfc3393 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:23:50.799565 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:23:50.800438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:23:50.800723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:23:50.801000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] PATH_INFO: `/v3/projects/cbfc99c4dc004ee69e0f6ccbf83917d9` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:23:50.802367 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbfc99c4dc004ee69e0f6ccbf83917d9)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:23:50.804193 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:23:50.855666 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbfc99c4dc004ee69e0f6ccbf83917d9', 'request_id': 'req-69a782b2-812b-4f75-bb33-5eb94c637fc1'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:50.861239 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbfc99c4dc004ee69e0f6ccbf83917d9', 'request_id': 'req-69a782b2-812b-4f75-bb33-5eb94c637fc1'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:50.867573 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbfc99c4dc004ee69e0f6ccbf83917d9', 'request_id': 'req-69a782b2-812b-4f75-bb33-5eb94c637fc1'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:50.881911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] The token cache is being invalidate because project cbfc99c4dc004ee69e0f6ccbf83917d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:23:50.882197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:50.882878 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-69a782b2-812b-4f75-bb33-5eb94c637fc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:23:50.884465 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1410/2789] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:23:50 2022] DELETE /identity/v3/projects/cbfc99c4dc004ee69e0f6ccbf83917d9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:24:00.497306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:24:00.878593 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:33.892645 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ca4d8-c2ac-43e1-8aae-6efccb56a28e service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:25:33.894111 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e97ca4d8-c2ac-43e1-8aae-6efccb56a28e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:25:34.043179 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ca4d8-c2ac-43e1-8aae-6efccb56a28e service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:25:34.119947 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e4769-502b-425e-b7b3-2847949ec696 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:25:34.120663 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e4769-502b-425e-b7b3-2847949ec696 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:25:34.120943 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e4769-502b-425e-b7b3-2847949ec696 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:25:34.121283 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e4769-502b-425e-b7b3-2847949ec696 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:25:34.123309 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2b8e4769-502b-425e-b7b3-2847949ec696 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:25:34.142052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e4769-502b-425e-b7b3-2847949ec696 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:25:34.144426 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e4769-502b-425e-b7b3-2847949ec696 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:25:34.156236 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1380/2790] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:25:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:25:36.934858 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14fec-8d17-425b-adcd-6975f01bf8d2 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:25:36.935397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14fec-8d17-425b-adcd-6975f01bf8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:25:36.935397 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14fec-8d17-425b-adcd-6975f01bf8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:25:36.954730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-d7d14fec-8d17-425b-adcd-6975f01bf8d2 None None] MFA Rules not processed for user `a895ab25adee48a3851ad15657c07a96`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:25:36.956252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d7d14fec-8d17-425b-adcd-6975f01bf8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:25:37.018106 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1411/2791] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:25:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:25:37.365580 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e4769-502b-425e-b7b3-2847949ec696 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:25:37.367421 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2b8e4769-502b-425e-b7b3-2847949ec696 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:25:37.407577 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=100 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:37.448253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e4769-502b-425e-b7b3-2847949ec696 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:25:37.452090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57853413-6ec4-47d3-a2f8-71c91f79f95e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:25:37.452735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57853413-6ec4-47d3-a2f8-71c91f79f95e service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:25:37.453046 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57853413-6ec4-47d3-a2f8-71c91f79f95e service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:25:37.453371 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57853413-6ec4-47d3-a2f8-71c91f79f95e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:25:37.455259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-57853413-6ec4-47d3-a2f8-71c91f79f95e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:25:37.466106 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57853413-6ec4-47d3-a2f8-71c91f79f95e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:25:37.467574 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57853413-6ec4-47d3-a2f8-71c91f79f95e service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:25:37.475999 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1381/2792] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:25:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:25:37.801826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14fec-8d17-425b-adcd-6975f01bf8d2 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:25:37.803499 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d7d14fec-8d17-425b-adcd-6975f01bf8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:25:37.874506 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14fec-8d17-425b-adcd-6975f01bf8d2 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:25:37.879407 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93107a-c98e-4af6-bed6-f305405a6c55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:25:37.880073 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93107a-c98e-4af6-bed6-f305405a6c55 service placement] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:25:37.880360 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93107a-c98e-4af6-bed6-f305405a6c55 service placement] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:25:37.880639 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93107a-c98e-4af6-bed6-f305405a6c55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:25:37.882452 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1a93107a-c98e-4af6-bed6-f305405a6c55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:25:37.893624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93107a-c98e-4af6-bed6-f305405a6c55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:25:37.895265 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93107a-c98e-4af6-bed6-f305405a6c55 service placement] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:25:37.905463 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1412/2793] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:25:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:25:45.126687 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57853413-6ec4-47d3-a2f8-71c91f79f95e service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:25:45.128405 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-57853413-6ec4-47d3-a2f8-71c91f79f95e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:25:45.159187 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57853413-6ec4-47d3-a2f8-71c91f79f95e service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:25:45.169619 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42194a87-095a-48dd-80c6-6b74a05db379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:25:45.170296 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42194a87-095a-48dd-80c6-6b74a05db379 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:25:45.170587 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42194a87-095a-48dd-80c6-6b74a05db379 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:25:45.170869 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42194a87-095a-48dd-80c6-6b74a05db379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:25:45.172926 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-42194a87-095a-48dd-80c6-6b74a05db379 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:25:45.186073 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42194a87-095a-48dd-80c6-6b74a05db379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:25:45.187939 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42194a87-095a-48dd-80c6-6b74a05db379 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:25:45.200369 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1382/2794] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:25:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:25:47.901192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=57 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:48.520747 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93107a-c98e-4af6-bed6-f305405a6c55 service placement] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:25:48.532919 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93107a-c98e-4af6-bed6-f305405a6c55 service placement] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:25:48.538326 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c9702-c7ae-47a5-bbf0-9095fba8235b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:25:48.538989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c9702-c7ae-47a5-bbf0-9095fba8235b service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:25:48.539271 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c9702-c7ae-47a5-bbf0-9095fba8235b service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:25:48.539603 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c9702-c7ae-47a5-bbf0-9095fba8235b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:25:48.541391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-fd9c9702-c7ae-47a5-bbf0-9095fba8235b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:25:48.553293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c9702-c7ae-47a5-bbf0-9095fba8235b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:25:48.554909 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c9702-c7ae-47a5-bbf0-9095fba8235b service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:25:48.565473 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1413/2795] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:25:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:25:55.195476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=28 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:58.561867 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=6 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:26:33.799345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42194a87-095a-48dd-80c6-6b74a05db379 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:26:33.821841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42194a87-095a-48dd-80c6-6b74a05db379 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:26:33.828286 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd23ef-4b09-4320-b1ee-c56177bf31d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:26:33.829295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd23ef-4b09-4320-b1ee-c56177bf31d7 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:26:33.829583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd23ef-4b09-4320-b1ee-c56177bf31d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:26:33.829858 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd23ef-4b09-4320-b1ee-c56177bf31d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:26:33.831861 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8ecd23ef-4b09-4320-b1ee-c56177bf31d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:26:33.856694 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd23ef-4b09-4320-b1ee-c56177bf31d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:26:33.858429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd23ef-4b09-4320-b1ee-c56177bf31d7 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:26:33.869073 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1383/2796] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:26:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:26:43.864819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=15 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:03.373907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c9702-c7ae-47a5-bbf0-9095fba8235b service nova] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:03.376051 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-fd9c9702-c7ae-47a5-bbf0-9095fba8235b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:03.413251 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c9702-c7ae-47a5-bbf0-9095fba8235b service nova] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:03.419267 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d416ca1-01a5-448f-b328-e154cf92e73e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:03.419913 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d416ca1-01a5-448f-b328-e154cf92e73e service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:03.420193 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d416ca1-01a5-448f-b328-e154cf92e73e service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:03.420461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d416ca1-01a5-448f-b328-e154cf92e73e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:03.422378 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6d416ca1-01a5-448f-b328-e154cf92e73e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:03.433208 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d416ca1-01a5-448f-b328-e154cf92e73e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:03.434908 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d416ca1-01a5-448f-b328-e154cf92e73e service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:03.444418 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1414/2797] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:27:13.440368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=15 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:24.753292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd23ef-4b09-4320-b1ee-c56177bf31d7 service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:24.754897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8ecd23ef-4b09-4320-b1ee-c56177bf31d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:24.767392 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd23ef-4b09-4320-b1ee-c56177bf31d7 service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:24.771865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bc950-17d2-4f3c-88c9-847fb92818d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:24.772525 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bc950-17d2-4f3c-88c9-847fb92818d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:24.772816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bc950-17d2-4f3c-88c9-847fb92818d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:24.773096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bc950-17d2-4f3c-88c9-847fb92818d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:24.775004 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0c2bc950-17d2-4f3c-88c9-847fb92818d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:24.863355 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2bc950-17d2-4f3c-88c9-847fb92818d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:24.864815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2bc950-17d2-4f3c-88c9-847fb92818d3 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:24.884346 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1384/2798] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:27:27.777588 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d416ca1-01a5-448f-b328-e154cf92e73e service nova] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:27.788794 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d416ca1-01a5-448f-b328-e154cf92e73e service nova] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:27.794604 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1c5e9-e471-41ad-a3b2-30ba92a98994 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:27.795334 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1c5e9-e471-41ad-a3b2-30ba92a98994 service placement] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:27.795624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1c5e9-e471-41ad-a3b2-30ba92a98994 service placement] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:27.795905 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1c5e9-e471-41ad-a3b2-30ba92a98994 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:27.797670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-0de1c5e9-e471-41ad-a3b2-30ba92a98994 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:27.810170 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1c5e9-e471-41ad-a3b2-30ba92a98994 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:27.811990 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1c5e9-e471-41ad-a3b2-30ba92a98994 service placement] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:27.822170 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1415/2799] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:27:27.846383 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bc950-17d2-4f3c-88c9-847fb92818d3 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:27.848017 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-0c2bc950-17d2-4f3c-88c9-847fb92818d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:27.862039 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bc950-17d2-4f3c-88c9-847fb92818d3 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:27.867588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f11e5f-b60d-4c4a-90b3-eb3f1361653c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:27.868496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f11e5f-b60d-4c4a-90b3-eb3f1361653c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:27.868780 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f11e5f-b60d-4c4a-90b3-eb3f1361653c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:27.869057 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f11e5f-b60d-4c4a-90b3-eb3f1361653c admin admin] PATH_INFO: `/v3/users/a895ab25adee48a3851ad15657c07a96` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:27.870398 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f11e5f-b60d-4c4a-90b3-eb3f1361653c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a895ab25adee48a3851ad15657c07a96)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:27.872347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f11e5f-b60d-4c4a-90b3-eb3f1361653c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:27.922144 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-91f11e5f-b60d-4c4a-90b3-eb3f1361653c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a895ab25adee48a3851ad15657c07a96'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:27.922917 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-91f11e5f-b60d-4c4a-90b3-eb3f1361653c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a895ab25adee48a3851ad15657c07a96'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:27.923524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-91f11e5f-b60d-4c4a-90b3-eb3f1361653c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a895ab25adee48a3851ad15657c07a96'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:27.931164 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-91f11e5f-b60d-4c4a-90b3-eb3f1361653c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a895ab25adee48a3851ad15657c07a96'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:27.943552 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-91f11e5f-b60d-4c4a-90b3-eb3f1361653c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a895ab25adee48a3851ad15657c07a96'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:27.958931 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1385/2800] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:27:27 2022] DELETE /identity/v3/users/a895ab25adee48a3851ad15657c07a96 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:27:28.147199 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1c5e9-e471-41ad-a3b2-30ba92a98994 service placement] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:28.161274 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1c5e9-e471-41ad-a3b2-30ba92a98994 service placement] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:28.219283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:28.220193 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:28.220479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:28.220756 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] PATH_INFO: `/v3/projects/46ff9242d2b147f690786e1284d1cfef` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:28.222066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46ff9242d2b147f690786e1284d1cfef)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:28.224769 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:28.277997 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46ff9242d2b147f690786e1284d1cfef', 'request_id': 'req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:28.283675 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46ff9242d2b147f690786e1284d1cfef', 'request_id': 'req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:28.289819 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46ff9242d2b147f690786e1284d1cfef', 'request_id': 'req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:28.304340 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] The token cache is being invalidate because project 46ff9242d2b147f690786e1284d1cfef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:27:28.304632 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:28.305303 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-6debacb1-a482-4cc8-aeb9-a6a1b1aeb528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:28.306934 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1416/2801] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:27:28 2022] DELETE /identity/v3/projects/46ff9242d2b147f690786e1284d1cfef => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:27:28.312092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd789-27c5-4786-b892-95d114a04ca6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:28.312441 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd789-27c5-4786-b892-95d114a04ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:28.312743 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd789-27c5-4786-b892-95d114a04ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:28.351453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-dfedd789-27c5-4786-b892-95d114a04ca6 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:28.352788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-dfedd789-27c5-4786-b892-95d114a04ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:28.372009 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:28.415943 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1386/2802] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:27:28.424087 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c17c02-09a1-42fe-9884-70f6306cb57a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:28.424388 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c17c02-09a1-42fe-9884-70f6306cb57a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:28.424668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c17c02-09a1-42fe-9884-70f6306cb57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:28.444800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-14c17c02-09a1-42fe-9884-70f6306cb57a None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:28.446035 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-14c17c02-09a1-42fe-9884-70f6306cb57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:28.454763 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1417/2803] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:28.459118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedd789-27c5-4786-b892-95d114a04ca6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:28.469084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedd789-27c5-4786-b892-95d114a04ca6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:28.474736 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f9329-d976-459e-8b14-ebf2d59f4529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:28.475476 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f9329-d976-459e-8b14-ebf2d59f4529 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:28.475937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f9329-d976-459e-8b14-ebf2d59f4529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:28.476219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f9329-d976-459e-8b14-ebf2d59f4529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:28.476983 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f9329-d976-459e-8b14-ebf2d59f4529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:27:28.477295 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f9329-d976-459e-8b14-ebf2d59f4529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:28.478951 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f9329-d976-459e-8b14-ebf2d59f4529 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:28.491009 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1387/2804] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:27:28.495366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c17c02-09a1-42fe-9884-70f6306cb57a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:28.504754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c17c02-09a1-42fe-9884-70f6306cb57a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:28.509846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104e9ec-7b53-435d-a9b8-26c5d783b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:28.510532 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e9ec-7b53-435d-a9b8-26c5d783b864 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:28.510815 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e9ec-7b53-435d-a9b8-26c5d783b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:28.511089 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e9ec-7b53-435d-a9b8-26c5d783b864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:28.511803 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104e9ec-7b53-435d-a9b8-26c5d783b864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:28.513733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104e9ec-7b53-435d-a9b8-26c5d783b864 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:28.543277 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1418/2805] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:27:28 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:27:28.547493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f9329-d976-459e-8b14-ebf2d59f4529 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:28.558128 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f9329-d976-459e-8b14-ebf2d59f4529 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:28.625886 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f023a6e-a1ec-4392-9e8a-5ea6704ccf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:28.626842 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f023a6e-a1ec-4392-9e8a-5ea6704ccf6c admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:28.627149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f023a6e-a1ec-4392-9e8a-5ea6704ccf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:28.627538 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f023a6e-a1ec-4392-9e8a-5ea6704ccf6c admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:28.628489 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f023a6e-a1ec-4392-9e8a-5ea6704ccf6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:28.631530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f023a6e-a1ec-4392-9e8a-5ea6704ccf6c admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:28.652831 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1388/2806] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:27:28 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:27:28.657321 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104e9ec-7b53-435d-a9b8-26c5d783b864 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:28.668117 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104e9ec-7b53-435d-a9b8-26c5d783b864 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:28.673121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62c597-8572-4a01-b37b-125f4127b1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:28.673789 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62c597-8572-4a01-b37b-125f4127b1fb admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:28.674085 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62c597-8572-4a01-b37b-125f4127b1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:28.674389 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62c597-8572-4a01-b37b-125f4127b1fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:28.675157 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62c597-8572-4a01-b37b-125f4127b1fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:28.676802 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62c597-8572-4a01-b37b-125f4127b1fb admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:28.687371 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1419/2807] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:27:28.691311 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f023a6e-a1ec-4392-9e8a-5ea6704ccf6c admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:28.702508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f023a6e-a1ec-4392-9e8a-5ea6704ccf6c admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:28.708693 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da8a8d-e3f8-47a2-ba38-c9e761976dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:28.709361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da8a8d-e3f8-47a2-ba38-c9e761976dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:28.709648 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da8a8d-e3f8-47a2-ba38-c9e761976dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:28.709942 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da8a8d-e3f8-47a2-ba38-c9e761976dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:28.710833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da8a8d-e3f8-47a2-ba38-c9e761976dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:28.712459 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da8a8d-e3f8-47a2-ba38-c9e761976dcb admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:28.723070 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1389/2808] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:27:28.727820 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62c597-8572-4a01-b37b-125f4127b1fb admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:28.742656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62c597-8572-4a01-b37b-125f4127b1fb admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:28.748007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903d745-8b5d-4b33-88fe-6bb3498415c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:28.748892 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903d745-8b5d-4b33-88fe-6bb3498415c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:28.749167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903d745-8b5d-4b33-88fe-6bb3498415c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:28.749442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903d745-8b5d-4b33-88fe-6bb3498415c5 admin admin] PATH_INFO: `/v3/projects/b282d40403ce422faee6e8e3e506d693/users/81db2c1797be4cdab4e35a7b3c1f3b5e/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:28.766481 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903d745-8b5d-4b33-88fe-6bb3498415c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b282d40403ce422faee6e8e3e506d693, user_id=81db2c1797be4cdab4e35a7b3c1f3b5e, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:28.770474 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903d745-8b5d-4b33-88fe-6bb3498415c5 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:28.786666 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1420/2809] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:27:28 2022] PUT /identity/v3/projects/b282d40403ce422faee6e8e3e506d693/users/81db2c1797be4cdab4e35a7b3c1f3b5e/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:27:28.797788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3771a-f5ae-49d4-b9f7-5a451a6b055d None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:28.798027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3771a-f5ae-49d4-b9f7-5a451a6b055d None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:28.798345 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3771a-f5ae-49d4-b9f7-5a451a6b055d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:28.826253 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-86c3771a-f5ae-49d4-b9f7-5a451a6b055d None None] MFA Rules not processed for user `81db2c1797be4cdab4e35a7b3c1f3b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:28.827850 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-86c3771a-f5ae-49d4-b9f7-5a451a6b055d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:28.893496 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1390/2810] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:28.898608 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903d745-8b5d-4b33-88fe-6bb3498415c5 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:28.913071 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:28.964716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903d745-8b5d-4b33-88fe-6bb3498415c5 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:28.969973 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9a927-a536-43a3-baaf-283992706977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:28.970701 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9a927-a536-43a3-baaf-283992706977 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:28.970980 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9a927-a536-43a3-baaf-283992706977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:28.971261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9a927-a536-43a3-baaf-283992706977 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:28.971945 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9a927-a536-43a3-baaf-283992706977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:28.973867 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9a927-a536-43a3-baaf-283992706977 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:29.002215 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1421/2811] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:27:28 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:27:29.006320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3771a-f5ae-49d4-b9f7-5a451a6b055d None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:29.017806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3771a-f5ae-49d4-b9f7-5a451a6b055d None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:29.031537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:29.077003 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2bbef-a230-43bd-b48b-9b70ce1f3f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:29.077970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bbef-a230-43bd-b48b-9b70ce1f3f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:29.078288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bbef-a230-43bd-b48b-9b70ce1f3f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:29.078567 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2bbef-a230-43bd-b48b-9b70ce1f3f01 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:29.079266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2bbef-a230-43bd-b48b-9b70ce1f3f01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:29.081225 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2bbef-a230-43bd-b48b-9b70ce1f3f01 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:29.101980 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1391/2812] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:27:29 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:27:29.106123 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9a927-a536-43a3-baaf-283992706977 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:29.116604 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9a927-a536-43a3-baaf-283992706977 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:29.123168 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4dd54-f472-4075-9a2f-017b23dce5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:29.123847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4dd54-f472-4075-9a2f-017b23dce5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:29.124130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4dd54-f472-4075-9a2f-017b23dce5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:29.124405 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4dd54-f472-4075-9a2f-017b23dce5c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:29.125136 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4dd54-f472-4075-9a2f-017b23dce5c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:29.126799 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4dd54-f472-4075-9a2f-017b23dce5c7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:29.138155 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1422/2813] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:27:29.142723 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2bbef-a230-43bd-b48b-9b70ce1f3f01 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:29.153361 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2bbef-a230-43bd-b48b-9b70ce1f3f01 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:29.159344 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56903c2-bff5-4342-b266-956ef88872aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:29.160252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56903c2-bff5-4342-b266-956ef88872aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:29.160504 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56903c2-bff5-4342-b266-956ef88872aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:29.160784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56903c2-bff5-4342-b266-956ef88872aa admin admin] PATH_INFO: `/v3/projects/6829e2bce94f41d49efc071c21e649ab/users/8169d8371aef4e7c9a35fa5ff8bd2ccb/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:29.178600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56903c2-bff5-4342-b266-956ef88872aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6829e2bce94f41d49efc071c21e649ab, user_id=8169d8371aef4e7c9a35fa5ff8bd2ccb, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:29.181281 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56903c2-bff5-4342-b266-956ef88872aa admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:29.196331 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1392/2814] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:27:29 2022] PUT /identity/v3/projects/6829e2bce94f41d49efc071c21e649ab/users/8169d8371aef4e7c9a35fa5ff8bd2ccb/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:27:29.200625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4dd54-f472-4075-9a2f-017b23dce5c7 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:29.211716 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4dd54-f472-4075-9a2f-017b23dce5c7 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:29.273797 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ae981-f35e-4e7a-a351-3d4279c2bcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:29.274628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ae981-f35e-4e7a-a351-3d4279c2bcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:29.274883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ae981-f35e-4e7a-a351-3d4279c2bcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:29.275159 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ae981-f35e-4e7a-a351-3d4279c2bcf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:29.275883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ae981-f35e-4e7a-a351-3d4279c2bcf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:29.277475 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ae981-f35e-4e7a-a351-3d4279c2bcf9 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:29.287054 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1423/2815] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:27:29.291458 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56903c2-bff5-4342-b266-956ef88872aa admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:29.302074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56903c2-bff5-4342-b266-956ef88872aa admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:29.307578 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee682f-655b-457a-8422-0b506c889154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:29.308482 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee682f-655b-457a-8422-0b506c889154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:29.308762 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee682f-655b-457a-8422-0b506c889154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:29.309092 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee682f-655b-457a-8422-0b506c889154 admin admin] PATH_INFO: `/v3/projects/6829e2bce94f41d49efc071c21e649ab/users/8169d8371aef4e7c9a35fa5ff8bd2ccb/roles/c184a177010144429f495f3290d28221` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:29.311445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee682f-655b-457a-8422-0b506c889154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6829e2bce94f41d49efc071c21e649ab, user_id=8169d8371aef4e7c9a35fa5ff8bd2ccb, role_id=c184a177010144429f495f3290d28221)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:29.314076 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee682f-655b-457a-8422-0b506c889154 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:29.324967 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1393/2816] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:27:29 2022] PUT /identity/v3/projects/6829e2bce94f41d49efc071c21e649ab/users/8169d8371aef4e7c9a35fa5ff8bd2ccb/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:27:29.335509 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f699032-f6dc-4347-ae2e-9c642f56294a None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:29.335730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f699032-f6dc-4347-ae2e-9c642f56294a None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:29.336002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f699032-f6dc-4347-ae2e-9c642f56294a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:29.361943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-6f699032-f6dc-4347-ae2e-9c642f56294a None None] MFA Rules not processed for user `8169d8371aef4e7c9a35fa5ff8bd2ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:29.363225 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-6f699032-f6dc-4347-ae2e-9c642f56294a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:29.433850 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1424/2817] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:29.445308 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee682f-655b-457a-8422-0b506c889154 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:29.447263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d2ee682f-655b-457a-8422-0b506c889154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:29.510365 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee682f-655b-457a-8422-0b506c889154 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:29.514994 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3c503-4c96-4fb2-9ab1-d7eefc1e8d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:29.515647 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3c503-4c96-4fb2-9ab1-d7eefc1e8d52 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:29.515930 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3c503-4c96-4fb2-9ab1-d7eefc1e8d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:29.516199 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3c503-4c96-4fb2-9ab1-d7eefc1e8d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:29.573843 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3c503-4c96-4fb2-9ab1-d7eefc1e8d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:29.575298 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3c503-4c96-4fb2-9ab1-d7eefc1e8d52 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:29.584505 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1394/2818] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:27:32.165323 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f699032-f6dc-4347-ae2e-9c642f56294a None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:32.175980 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f699032-f6dc-4347-ae2e-9c642f56294a None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:32.179865 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4c46a-1233-471a-aa54-7d5afeb86b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:32.180535 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4c46a-1233-471a-aa54-7d5afeb86b54 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:32.180823 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4c46a-1233-471a-aa54-7d5afeb86b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:32.181110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4c46a-1233-471a-aa54-7d5afeb86b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:32.188078 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:32.190633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4c46a-1233-471a-aa54-7d5afeb86b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:32.192568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4c46a-1233-471a-aa54-7d5afeb86b54 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:32.203169 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1425/2819] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:27:39.580782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=84,keystone:INSERT=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:42.200266 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:44.877499 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3c503-4c96-4fb2-9ab1-d7eefc1e8d52 service cinder] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:44.939937 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3c503-4c96-4fb2-9ab1-d7eefc1e8d52 service cinder] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:44.945486 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:44.946429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:44.946717 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:44.946993 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] PATH_INFO: `/v3/users/81db2c1797be4cdab4e35a7b3c1f3b5e` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:44.948307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81db2c1797be4cdab4e35a7b3c1f3b5e)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:44.950306 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:45.003452 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81db2c1797be4cdab4e35a7b3c1f3b5e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.004185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81db2c1797be4cdab4e35a7b3c1f3b5e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.004894 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81db2c1797be4cdab4e35a7b3c1f3b5e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.012261 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81db2c1797be4cdab4e35a7b3c1f3b5e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.025030 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81db2c1797be4cdab4e35a7b3c1f3b5e'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.040570 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1395/2820] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:27:44 2022] DELETE /identity/v3/users/81db2c1797be4cdab4e35a7b3c1f3b5e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:27:45.045647 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594e1b-ff55-4bd2-a227-a7239c186a8b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:45.046171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594e1b-ff55-4bd2-a227-a7239c186a8b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:45.046297 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594e1b-ff55-4bd2-a227-a7239c186a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:45.065998 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-c8594e1b-ff55-4bd2-a227-a7239c186a8b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:45.067230 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c8594e1b-ff55-4bd2-a227-a7239c186a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:45.128287 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1426/2821] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:45.138010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:45.139598 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:45.210547 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3ef34-77b7-46ac-bd61-b16d237f236f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:45.214589 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1320c-4f30-486d-9c08-a9edfc9b95df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:45.215239 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1320c-4f30-486d-9c08-a9edfc9b95df service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:45.215522 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1320c-4f30-486d-9c08-a9edfc9b95df service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:45.215795 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1320c-4f30-486d-9c08-a9edfc9b95df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:45.225619 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1320c-4f30-486d-9c08-a9edfc9b95df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:45.227091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1320c-4f30-486d-9c08-a9edfc9b95df service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:45.236773 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1396/2822] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:27:45.522438 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8594e1b-ff55-4bd2-a227-a7239c186a8b None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:45.524003 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c8594e1b-ff55-4bd2-a227-a7239c186a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:45.537077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8594e1b-ff55-4bd2-a227-a7239c186a8b None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:45.541828 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:45.542719 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:45.542998 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:45.543268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] PATH_INFO: `/v3/projects/b282d40403ce422faee6e8e3e506d693` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:45.544541 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b282d40403ce422faee6e8e3e506d693)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:45.546395 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:45.595575 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b282d40403ce422faee6e8e3e506d693', 'request_id': 'req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.600907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b282d40403ce422faee6e8e3e506d693', 'request_id': 'req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.606956 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b282d40403ce422faee6e8e3e506d693', 'request_id': 'req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.620885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] The token cache is being invalidate because project b282d40403ce422faee6e8e3e506d693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:27:45.621181 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.621804 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.623394 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1427/2823] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:27:45 2022] DELETE /identity/v3/projects/b282d40403ce422faee6e8e3e506d693 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:27:45.627488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1320c-4f30-486d-9c08-a9edfc9b95df service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:45.637565 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1320c-4f30-486d-9c08-a9edfc9b95df service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:45.657056 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:45.697681 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54aa367-d6a4-4529-aaf4-aaaeb28d49b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:45.698590 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54aa367-d6a4-4529-aaf4-aaaeb28d49b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:45.698877 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54aa367-d6a4-4529-aaf4-aaaeb28d49b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:45.699157 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54aa367-d6a4-4529-aaf4-aaaeb28d49b2 admin admin] PATH_INFO: `/v3/users/8169d8371aef4e7c9a35fa5ff8bd2ccb` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:45.700459 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54aa367-d6a4-4529-aaf4-aaaeb28d49b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8169d8371aef4e7c9a35fa5ff8bd2ccb)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:45.702817 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54aa367-d6a4-4529-aaf4-aaaeb28d49b2 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:45.759804 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b54aa367-d6a4-4529-aaf4-aaaeb28d49b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8169d8371aef4e7c9a35fa5ff8bd2ccb'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.760581 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b54aa367-d6a4-4529-aaf4-aaaeb28d49b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8169d8371aef4e7c9a35fa5ff8bd2ccb'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.761396 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b54aa367-d6a4-4529-aaf4-aaaeb28d49b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8169d8371aef4e7c9a35fa5ff8bd2ccb'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.771222 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b54aa367-d6a4-4529-aaf4-aaaeb28d49b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8169d8371aef4e7c9a35fa5ff8bd2ccb'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.784533 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-b54aa367-d6a4-4529-aaf4-aaaeb28d49b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8169d8371aef4e7c9a35fa5ff8bd2ccb'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:45.799148 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1397/2824] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:27:45 2022] DELETE /identity/v3/users/8169d8371aef4e7c9a35fa5ff8bd2ccb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:27:46.081073 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:46.083252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:46.098449 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c8139-f7f8-40a7-bec4-9461dc2098f8 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:46.157060 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:46.157850 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:46.158130 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:46.158434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] PATH_INFO: `/v3/projects/6829e2bce94f41d49efc071c21e649ab` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:46.159622 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6829e2bce94f41d49efc071c21e649ab)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:46.161536 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:46.201320 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:46.220116 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6829e2bce94f41d49efc071c21e649ab', 'request_id': 'req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:46.229133 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6829e2bce94f41d49efc071c21e649ab', 'request_id': 'req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:46.235341 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6829e2bce94f41d49efc071c21e649ab', 'request_id': 'req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:46.250612 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] The token cache is being invalidate because project 6829e2bce94f41d49efc071c21e649ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:27:46.250900 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:46.251577 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-c192469f-1c66-4ad6-9d6e-9ceed1fa53d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:46.253247 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1428/2825] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:27:46 2022] DELETE /identity/v3/projects/6829e2bce94f41d49efc071c21e649ab => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:27:46.258485 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1abbe-e4aa-47a4-b9ce-a76165d57270 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:46.258984 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1abbe-e4aa-47a4-b9ce-a76165d57270 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:46.259088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1abbe-e4aa-47a4-b9ce-a76165d57270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:46.278360 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-56f1abbe-e4aa-47a4-b9ce-a76165d57270 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:46.279579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-56f1abbe-e4aa-47a4-b9ce-a76165d57270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:46.340071 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1398/2826] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:46.348222 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b4f2d-553f-4fbb-b9a0-3c113ed7966e None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:46.348504 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b4f2d-553f-4fbb-b9a0-3c113ed7966e None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:46.348785 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b4f2d-553f-4fbb-b9a0-3c113ed7966e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:46.367976 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-a78b4f2d-553f-4fbb-b9a0-3c113ed7966e None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:46.369189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a78b4f2d-553f-4fbb-b9a0-3c113ed7966e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:46.378177 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1429/2827] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:46.382769 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1abbe-e4aa-47a4-b9ce-a76165d57270 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:46.393294 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1abbe-e4aa-47a4-b9ce-a76165d57270 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:46.399263 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a1289-4354-4bb9-8a97-2cd4de8d4951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:46.399945 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1289-4354-4bb9-8a97-2cd4de8d4951 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:46.400231 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1289-4354-4bb9-8a97-2cd4de8d4951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:46.400503 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a1289-4354-4bb9-8a97-2cd4de8d4951 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:46.401242 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a1289-4354-4bb9-8a97-2cd4de8d4951 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:27:46.401549 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a1289-4354-4bb9-8a97-2cd4de8d4951 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:46.403247 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a1289-4354-4bb9-8a97-2cd4de8d4951 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:46.415772 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1399/2828] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:27:46.420121 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b4f2d-553f-4fbb-b9a0-3c113ed7966e None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:46.429653 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b4f2d-553f-4fbb-b9a0-3c113ed7966e None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:46.434733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb6d90-50b2-4a7f-a986-219d172cda1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:46.435384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb6d90-50b2-4a7f-a986-219d172cda1b admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:46.435680 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb6d90-50b2-4a7f-a986-219d172cda1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:46.435945 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb6d90-50b2-4a7f-a986-219d172cda1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:46.436634 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb6d90-50b2-4a7f-a986-219d172cda1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:46.438941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb6d90-50b2-4a7f-a986-219d172cda1b admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:46.467690 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1430/2829] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:27:46 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:27:46.471930 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a1289-4354-4bb9-8a97-2cd4de8d4951 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:46.482024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a1289-4354-4bb9-8a97-2cd4de8d4951 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:46.513676 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:46.549197 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a74a7a-aa15-4025-8ab7-413b36a04a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:46.549870 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a74a7a-aa15-4025-8ab7-413b36a04a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:46.550149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a74a7a-aa15-4025-8ab7-413b36a04a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:46.550466 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a74a7a-aa15-4025-8ab7-413b36a04a81 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:46.551156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a74a7a-aa15-4025-8ab7-413b36a04a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:46.553105 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a74a7a-aa15-4025-8ab7-413b36a04a81 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:46.575286 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1400/2830] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:27:46 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:27:46.581254 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb6d90-50b2-4a7f-a986-219d172cda1b admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:46.594874 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb6d90-50b2-4a7f-a986-219d172cda1b admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:46.600666 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c31ad5-065b-430a-9919-21e7dce9a072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:46.601348 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c31ad5-065b-430a-9919-21e7dce9a072 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:46.601627 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c31ad5-065b-430a-9919-21e7dce9a072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:46.601907 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c31ad5-065b-430a-9919-21e7dce9a072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:46.602710 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c31ad5-065b-430a-9919-21e7dce9a072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:46.604368 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c31ad5-065b-430a-9919-21e7dce9a072 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:46.616022 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1431/2831] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:27:46.620275 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a74a7a-aa15-4025-8ab7-413b36a04a81 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:46.630881 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a74a7a-aa15-4025-8ab7-413b36a04a81 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:46.636146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ca863-4c5b-478c-9d31-f1f093be0428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:46.636815 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ca863-4c5b-478c-9d31-f1f093be0428 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:46.637100 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ca863-4c5b-478c-9d31-f1f093be0428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:46.637379 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ca863-4c5b-478c-9d31-f1f093be0428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:46.638115 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ca863-4c5b-478c-9d31-f1f093be0428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:46.639887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ca863-4c5b-478c-9d31-f1f093be0428 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:46.650362 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1401/2832] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:27:46.654263 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c31ad5-065b-430a-9919-21e7dce9a072 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:46.664478 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c31ad5-065b-430a-9919-21e7dce9a072 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:46.669607 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc73f6b-9bf4-436f-baad-efe51dbb81cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:46.670461 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc73f6b-9bf4-436f-baad-efe51dbb81cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:46.670730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc73f6b-9bf4-436f-baad-efe51dbb81cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:46.671002 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc73f6b-9bf4-436f-baad-efe51dbb81cb admin admin] PATH_INFO: `/v3/projects/383c91702a6e41359f1cea6025b880a3/users/86fab3e09458469babcf2707954f8d85/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:46.685739 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc73f6b-9bf4-436f-baad-efe51dbb81cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=383c91702a6e41359f1cea6025b880a3, user_id=86fab3e09458469babcf2707954f8d85, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:46.688613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc73f6b-9bf4-436f-baad-efe51dbb81cb admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:46.704623 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1432/2833] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:27:46 2022] PUT /identity/v3/projects/383c91702a6e41359f1cea6025b880a3/users/86fab3e09458469babcf2707954f8d85/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:27:46.721010 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342e177-402f-4747-bb97-0992df088053 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:46.721307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342e177-402f-4747-bb97-0992df088053 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:46.721584 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342e177-402f-4747-bb97-0992df088053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:46.751167 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-8342e177-402f-4747-bb97-0992df088053 None None] MFA Rules not processed for user `86fab3e09458469babcf2707954f8d85`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:46.752411 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8342e177-402f-4747-bb97-0992df088053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:46.818726 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1402/2834] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:46.831042 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc73f6b-9bf4-436f-baad-efe51dbb81cb admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:46.833967 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4dc73f6b-9bf4-436f-baad-efe51dbb81cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:46.895264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc73f6b-9bf4-436f-baad-efe51dbb81cb admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:46.899987 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46187c32-ae2e-4bc0-af24-58e16f2d1c18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:46.900611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46187c32-ae2e-4bc0-af24-58e16f2d1c18 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:46.900893 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46187c32-ae2e-4bc0-af24-58e16f2d1c18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:46.901194 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46187c32-ae2e-4bc0-af24-58e16f2d1c18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:46.910323 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46187c32-ae2e-4bc0-af24-58e16f2d1c18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:46.911827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46187c32-ae2e-4bc0-af24-58e16f2d1c18 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:46.920872 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1433/2835] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:27:46.940630 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342e177-402f-4747-bb97-0992df088053 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:47.008904 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342e177-402f-4747-bb97-0992df088053 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:47.013960 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:47.014979 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:47.015260 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:47.015593 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] PATH_INFO: `/v3/users/86fab3e09458469babcf2707954f8d85` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:47.016832 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] RBAC: Authorizing `identity:delete_user(user_id=86fab3e09458469babcf2707954f8d85)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:47.018884 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:47.063149 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:47.070050 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86fab3e09458469babcf2707954f8d85'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:47.071080 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86fab3e09458469babcf2707954f8d85'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:47.071749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86fab3e09458469babcf2707954f8d85'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:47.079404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86fab3e09458469babcf2707954f8d85'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:47.091185 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86fab3e09458469babcf2707954f8d85'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:47.105139 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1403/2836] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:27:46 2022] DELETE /identity/v3/users/86fab3e09458469babcf2707954f8d85 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:27:47.109859 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deb3f3-51d0-4564-a46a-1c2496c6446c None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:47.110390 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deb3f3-51d0-4564-a46a-1c2496c6446c None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:47.110512 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deb3f3-51d0-4564-a46a-1c2496c6446c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:47.127289 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-56deb3f3-51d0-4564-a46a-1c2496c6446c None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:47.128472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-56deb3f3-51d0-4564-a46a-1c2496c6446c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:47.177743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:47.192395 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1434/2837] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:47.204096 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:47.205568 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:47.261692 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de170dd1-b70d-4a05-adc9-56580f4cf7da admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:47.265749 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a60f7-8475-45ae-816f-eadb7538a883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:47.266453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a60f7-8475-45ae-816f-eadb7538a883 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:47.266857 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a60f7-8475-45ae-816f-eadb7538a883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:47.267302 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a60f7-8475-45ae-816f-eadb7538a883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:47.284123 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a60f7-8475-45ae-816f-eadb7538a883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:47.285623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a60f7-8475-45ae-816f-eadb7538a883 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:47.295692 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1404/2838] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:27:47.575877 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56deb3f3-51d0-4564-a46a-1c2496c6446c None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:47.577725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-56deb3f3-51d0-4564-a46a-1c2496c6446c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:47.590911 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56deb3f3-51d0-4564-a46a-1c2496c6446c None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:47.596453 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:47.597366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:47.597652 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:47.597927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] PATH_INFO: `/v3/projects/383c91702a6e41359f1cea6025b880a3` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:47.599352 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=383c91702a6e41359f1cea6025b880a3)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:47.601225 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:47.648020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383c91702a6e41359f1cea6025b880a3', 'request_id': 'req-ec62fab6-d610-49de-be1a-0d4bfaf355f7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:47.653332 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383c91702a6e41359f1cea6025b880a3', 'request_id': 'req-ec62fab6-d610-49de-be1a-0d4bfaf355f7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:47.659387 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383c91702a6e41359f1cea6025b880a3', 'request_id': 'req-ec62fab6-d610-49de-be1a-0d4bfaf355f7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:47.673495 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] The token cache is being invalidate because project 383c91702a6e41359f1cea6025b880a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:27:47.673784 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:47.674444 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-ec62fab6-d610-49de-be1a-0d4bfaf355f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:27:47.676027 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1435/2839] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:27:47 2022] DELETE /identity/v3/projects/383c91702a6e41359f1cea6025b880a3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:27:47.681069 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a204af-97ab-4017-82d6-a462addeaee4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:47.681360 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a204af-97ab-4017-82d6-a462addeaee4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:47.681643 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a204af-97ab-4017-82d6-a462addeaee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:47.700323 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-60a204af-97ab-4017-82d6-a462addeaee4 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:47.701788 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-60a204af-97ab-4017-82d6-a462addeaee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:47.761923 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1405/2840] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:47.770004 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109f0e4-c624-49b5-bdbc-1b5067f4d9eb None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:47.770310 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109f0e4-c624-49b5-bdbc-1b5067f4d9eb None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:47.770591 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109f0e4-c624-49b5-bdbc-1b5067f4d9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:47.788935 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-4109f0e4-c624-49b5-bdbc-1b5067f4d9eb None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:47.790146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4109f0e4-c624-49b5-bdbc-1b5067f4d9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:47.798669 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1436/2841] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:47.803036 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a204af-97ab-4017-82d6-a462addeaee4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:47.813536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a204af-97ab-4017-82d6-a462addeaee4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:47.819237 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fd570-e662-48b4-8178-c6e8b57bf5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:47.819890 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fd570-e662-48b4-8178-c6e8b57bf5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:47.820173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fd570-e662-48b4-8178-c6e8b57bf5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:47.820447 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fd570-e662-48b4-8178-c6e8b57bf5f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:47.821173 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fd570-e662-48b4-8178-c6e8b57bf5f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:27:47.821583 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fd570-e662-48b4-8178-c6e8b57bf5f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:47.823210 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fd570-e662-48b4-8178-c6e8b57bf5f8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:47.836057 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1406/2842] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:27:47.840965 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109f0e4-c624-49b5-bdbc-1b5067f4d9eb None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:47.855242 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109f0e4-c624-49b5-bdbc-1b5067f4d9eb None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:47.863020 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305faf-2c8e-476f-b54b-1f87a5d9a536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:47.863663 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305faf-2c8e-476f-b54b-1f87a5d9a536 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:47.863970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305faf-2c8e-476f-b54b-1f87a5d9a536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:47.864246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305faf-2c8e-476f-b54b-1f87a5d9a536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:47.864924 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49305faf-2c8e-476f-b54b-1f87a5d9a536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:47.866885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49305faf-2c8e-476f-b54b-1f87a5d9a536 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:47.895394 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1437/2843] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:27:47 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:27:47.899999 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fd570-e662-48b4-8178-c6e8b57bf5f8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:47.910422 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fd570-e662-48b4-8178-c6e8b57bf5f8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:47.955250 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:47.970554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db892b1c-ad73-4d2a-8cee-4800281e5019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:47.971236 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db892b1c-ad73-4d2a-8cee-4800281e5019 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:47.971517 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db892b1c-ad73-4d2a-8cee-4800281e5019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:47.971794 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db892b1c-ad73-4d2a-8cee-4800281e5019 admin admin] PATH_INFO: `/v3/users` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:47.972496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db892b1c-ad73-4d2a-8cee-4800281e5019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:47.974445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db892b1c-ad73-4d2a-8cee-4800281e5019 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:47.996411 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1407/2844] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:27:47 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:27:48.000721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305faf-2c8e-476f-b54b-1f87a5d9a536 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:48.011287 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305faf-2c8e-476f-b54b-1f87a5d9a536 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:48.016244 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c228c-632e-4737-a6ea-bf0649741ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:48.016910 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c228c-632e-4737-a6ea-bf0649741ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:48.017188 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c228c-632e-4737-a6ea-bf0649741ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:48.017465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c228c-632e-4737-a6ea-bf0649741ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:48.018188 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c228c-632e-4737-a6ea-bf0649741ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:48.019822 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c228c-632e-4737-a6ea-bf0649741ee8 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:48.030328 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1438/2845] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:27:48.034343 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db892b1c-ad73-4d2a-8cee-4800281e5019 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:48.044514 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db892b1c-ad73-4d2a-8cee-4800281e5019 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:48.049806 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe6da6-d261-4d70-92ec-f301f69c5bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:48.050486 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe6da6-d261-4d70-92ec-f301f69c5bec admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:48.050773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe6da6-d261-4d70-92ec-f301f69c5bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:48.051078 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe6da6-d261-4d70-92ec-f301f69c5bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:48.051812 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe6da6-d261-4d70-92ec-f301f69c5bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:48.053407 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe6da6-d261-4d70-92ec-f301f69c5bec admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:48.063115 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1408/2846] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:27:48.068870 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c228c-632e-4737-a6ea-bf0649741ee8 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:48.081249 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c228c-632e-4737-a6ea-bf0649741ee8 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:48.089837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb5855-ace8-4064-8f79-f58cce415dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:48.091223 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5855-ace8-4064-8f79-f58cce415dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:48.091628 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5855-ace8-4064-8f79-f58cce415dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:48.092010 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5855-ace8-4064-8f79-f58cce415dbc admin admin] PATH_INFO: `/v3/projects/dccedf0b40e94111bf3b31f9347dfa44/users/7715154f39dc457695d5991dac30e107/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:48.108322 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb5855-ace8-4064-8f79-f58cce415dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dccedf0b40e94111bf3b31f9347dfa44, user_id=7715154f39dc457695d5991dac30e107, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:48.111016 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb5855-ace8-4064-8f79-f58cce415dbc admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:48.127069 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1439/2847] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:27:48 2022] PUT /identity/v3/projects/dccedf0b40e94111bf3b31f9347dfa44/users/7715154f39dc457695d5991dac30e107/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:27:48.132259 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c451-31f4-4b14-a40c-039fb0f413f1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:48.132545 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c451-31f4-4b14-a40c-039fb0f413f1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:48.132820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c451-31f4-4b14-a40c-039fb0f413f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:48.150833 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e225c451-31f4-4b14-a40c-039fb0f413f1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:48.152019 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e225c451-31f4-4b14-a40c-039fb0f413f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:48.211539 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1409/2848] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:48.223401 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb5855-ace8-4064-8f79-f58cce415dbc admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:48.225391 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-53eb5855-ace8-4064-8f79-f58cce415dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:48.291105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb5855-ace8-4064-8f79-f58cce415dbc admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:48.295625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26686304-e418-4980-9ebf-c24be4b38e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:48.296283 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26686304-e418-4980-9ebf-c24be4b38e71 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:48.296566 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26686304-e418-4980-9ebf-c24be4b38e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:48.296839 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26686304-e418-4980-9ebf-c24be4b38e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:48.306210 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26686304-e418-4980-9ebf-c24be4b38e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:48.307680 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26686304-e418-4980-9ebf-c24be4b38e71 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:48.317273 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1440/2849] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:27:48.932623 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0005-6420-406e-9ed6-9021392979bd None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:48.933152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0005-6420-406e-9ed6-9021392979bd None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:48.933152 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0005-6420-406e-9ed6-9021392979bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:48.961074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-265d0005-6420-406e-9ed6-9021392979bd None None] MFA Rules not processed for user `7715154f39dc457695d5991dac30e107`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:48.962328 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-265d0005-6420-406e-9ed6-9021392979bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:49.031238 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1410/2850] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:49.042537 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26686304-e418-4980-9ebf-c24be4b38e71 service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:49.044192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-26686304-e418-4980-9ebf-c24be4b38e71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:49.063302 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:49.124863 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26686304-e418-4980-9ebf-c24be4b38e71 service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:49.128670 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40df524-7204-4733-9eb5-fbaf517cd2c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:49.129333 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40df524-7204-4733-9eb5-fbaf517cd2c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:49.129615 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40df524-7204-4733-9eb5-fbaf517cd2c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:49.129890 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40df524-7204-4733-9eb5-fbaf517cd2c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:49.138423 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40df524-7204-4733-9eb5-fbaf517cd2c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:49.139857 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40df524-7204-4733-9eb5-fbaf517cd2c3 service cinder] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:49.148612 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1441/2851] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:27:50.506882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f322-f497-425c-9d2c-f879fe2ff0a4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:50.507409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f322-f497-425c-9d2c-f879fe2ff0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:50.507558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f322-f497-425c-9d2c-f879fe2ff0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:50.527462 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7f39f322-f497-425c-9d2c-f879fe2ff0a4 None None] MFA Rules not processed for user `7715154f39dc457695d5991dac30e107`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:27:50.528682 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7f39f322-f497-425c-9d2c-f879fe2ff0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:50.536776 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1411/2852] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:27:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:27:50.576099 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40df524-7204-4733-9eb5-fbaf517cd2c3 service cinder] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:50.578007 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-a40df524-7204-4733-9eb5-fbaf517cd2c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:50.660629 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40df524-7204-4733-9eb5-fbaf517cd2c3 service cinder] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:50.665093 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254318a-5c40-48c0-9a23-167fc014aa3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b97736e49e2a4bfb9cf7093cfa8b2b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:50.665759 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254318a-5c40-48c0-9a23-167fc014aa3d service glance] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:50.666038 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254318a-5c40-48c0-9a23-167fc014aa3d service glance] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:50.666358 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254318a-5c40-48c0-9a23-167fc014aa3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:50.668161 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7254318a-5c40-48c0-9a23-167fc014aa3d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:50.741939 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254318a-5c40-48c0-9a23-167fc014aa3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:50.743492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254318a-5c40-48c0-9a23-167fc014aa3d service glance] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:50.757670 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1442/2853] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:27:51.303929 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39f322-f497-425c-9d2c-f879fe2ff0a4 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:27:51.313713 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39f322-f497-425c-9d2c-f879fe2ff0a4 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:27:51.317192 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735f35e-71ec-4537-990c-22624e41e991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:27:51.317839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735f35e-71ec-4537-990c-22624e41e991 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:27:51.318118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735f35e-71ec-4537-990c-22624e41e991 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:27:51.318432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735f35e-71ec-4537-990c-22624e41e991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:27:51.320352 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8735f35e-71ec-4537-990c-22624e41e991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:27:51.334252 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735f35e-71ec-4537-990c-22624e41e991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:27:51.335808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735f35e-71ec-4537-990c-22624e41e991 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:27:51.346966 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1412/2854] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:27:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:28:00.751232 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=91 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:01.342292 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:23.391857 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497b379-e18d-482c-bd03-c462b4cc8867 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:23.392748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497b379-e18d-482c-bd03-c462b4cc8867 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:23.392748 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497b379-e18d-482c-bd03-c462b4cc8867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:23.441552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-d497b379-e18d-482c-bd03-c462b4cc8867 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:28:23.442923 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d497b379-e18d-482c-bd03-c462b4cc8867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:28:23.452471 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1443/2855] 10.222.0.39 () {58 vars in 1016 bytes} [Tue Jul 5 10:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:28:23.479519 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735f35e-71ec-4537-990c-22624e41e991 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:28:23.481408 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-8735f35e-71ec-4537-990c-22624e41e991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:28:23.495642 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735f35e-71ec-4537-990c-22624e41e991 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:28:23.501652 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1b445-ef22-4a57-a226-77940c27dec7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:28:23.502341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1b445-ef22-4a57-a226-77940c27dec7 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:23.502624 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1b445-ef22-4a57-a226-77940c27dec7 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:23.502897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1b445-ef22-4a57-a226-77940c27dec7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:23.511091 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1b445-ef22-4a57-a226-77940c27dec7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:28:23.512707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1b445-ef22-4a57-a226-77940c27dec7 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:28:23.522305 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1413/2856] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:28:27.487319 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18471a26-ad6d-4909-98e3-4d2833a79773 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:27.487898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18471a26-ad6d-4909-98e3-4d2833a79773 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:27.487898 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18471a26-ad6d-4909-98e3-4d2833a79773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:27.507547 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-18471a26-ad6d-4909-98e3-4d2833a79773 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:28:27.508759 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-18471a26-ad6d-4909-98e3-4d2833a79773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:28:27.517945 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1444/2857] 10.222.0.39 () {58 vars in 1016 bytes} [Tue Jul 5 10:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:28:27.540035 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1b445-ef22-4a57-a226-77940c27dec7 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:28:27.542247 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-38c1b445-ef22-4a57-a226-77940c27dec7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:28:27.562208 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1b445-ef22-4a57-a226-77940c27dec7 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:28:27.569216 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5e67d-ff3c-4590-8b63-00aaea06e849 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:28:27.569905 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5e67d-ff3c-4590-8b63-00aaea06e849 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:27.570183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5e67d-ff3c-4590-8b63-00aaea06e849 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:27.570486 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5e67d-ff3c-4590-8b63-00aaea06e849 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:27.581554 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5e67d-ff3c-4590-8b63-00aaea06e849 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:28:27.583200 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5e67d-ff3c-4590-8b63-00aaea06e849 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:28:27.603606 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1414/2858] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:28:35.241709 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18471a26-ad6d-4909-98e3-4d2833a79773 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:28:35.245270 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-18471a26-ad6d-4909-98e3-4d2833a79773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:28:35.258501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18471a26-ad6d-4909-98e3-4d2833a79773 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:28:35.263851 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5aa51-88fa-48b4-aa57-9082ba8cccfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:28:35.264597 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5aa51-88fa-48b4-aa57-9082ba8cccfb service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:35.264902 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5aa51-88fa-48b4-aa57-9082ba8cccfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:35.265204 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5aa51-88fa-48b4-aa57-9082ba8cccfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:35.267323 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-c7a5aa51-88fa-48b4-aa57-9082ba8cccfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:28:35.277725 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5aa51-88fa-48b4-aa57-9082ba8cccfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:28:35.279203 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5aa51-88fa-48b4-aa57-9082ba8cccfb service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:28:35.287818 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1445/2859] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:28:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:28:36.958488 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f396cb6-f984-4180-88eb-f8eaa93da7a1 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:36.959084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f396cb6-f984-4180-88eb-f8eaa93da7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:36.959084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f396cb6-f984-4180-88eb-f8eaa93da7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:36.978184 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-1f396cb6-f984-4180-88eb-f8eaa93da7a1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:28:36.979561 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-1f396cb6-f984-4180-88eb-f8eaa93da7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:28:36.988878 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1415/2860] 10.222.0.39 () {58 vars in 1016 bytes} [Tue Jul 5 10:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:28:37.013904 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5aa51-88fa-48b4-aa57-9082ba8cccfb service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:28:37.027910 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5aa51-88fa-48b4-aa57-9082ba8cccfb service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:28:37.034259 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f4fad-3614-4192-8783-c043f3ddc2ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:28:37.034975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f4fad-3614-4192-8783-c043f3ddc2ad service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:37.035246 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f4fad-3614-4192-8783-c043f3ddc2ad service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:37.035515 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f4fad-3614-4192-8783-c043f3ddc2ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:37.044491 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f4fad-3614-4192-8783-c043f3ddc2ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:28:37.046089 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f4fad-3614-4192-8783-c043f3ddc2ad service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:28:37.056003 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1446/2861] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:28:40.081735 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0eb76f-4cfa-45b8-aefd-dd90e9d6ecc3 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:40.082285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0eb76f-4cfa-45b8-aefd-dd90e9d6ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:40.082285 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0eb76f-4cfa-45b8-aefd-dd90e9d6ecc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:40.101842 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-7b0eb76f-4cfa-45b8-aefd-dd90e9d6ecc3 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:28:40.103081 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7b0eb76f-4cfa-45b8-aefd-dd90e9d6ecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:28:40.112162 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1416/2862] 10.222.0.39 () {58 vars in 1016 bytes} [Tue Jul 5 10:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:28:40.136580 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f4fad-3614-4192-8783-c043f3ddc2ad service nova] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:28:40.148793 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f4fad-3614-4192-8783-c043f3ddc2ad service nova] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:28:40.156090 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9368ecc-221f-46c1-9235-647a23e589ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:28:40.156860 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9368ecc-221f-46c1-9235-647a23e589ac service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:40.157147 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9368ecc-221f-46c1-9235-647a23e589ac service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:40.157426 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9368ecc-221f-46c1-9235-647a23e589ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:40.166354 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9368ecc-221f-46c1-9235-647a23e589ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:28:40.167968 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9368ecc-221f-46c1-9235-647a23e589ac service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:28:40.177985 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1447/2863] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:28:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:28:44.314896 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbf482-3c63-40eb-a185-c47a172721d4 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:44.315723 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbf482-3c63-40eb-a185-c47a172721d4 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:44.315923 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbf482-3c63-40eb-a185-c47a172721d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:44.346347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-45cbf482-3c63-40eb-a185-c47a172721d4 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:28:44.347602 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-45cbf482-3c63-40eb-a185-c47a172721d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:28:44.357107 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1417/2864] 10.222.0.39 () {58 vars in 1016 bytes} [Tue Jul 5 10:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:28:44.378983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9368ecc-221f-46c1-9235-647a23e589ac service nova] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:28:44.391743 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9368ecc-221f-46c1-9235-647a23e589ac service nova] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:28:44.398933 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05d5d5-0443-4afd-8255-fc72c02ee34d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:28:44.399583 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d5d5-0443-4afd-8255-fc72c02ee34d service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:44.399860 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d5d5-0443-4afd-8255-fc72c02ee34d service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:44.400144 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d5d5-0443-4afd-8255-fc72c02ee34d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:44.409097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05d5d5-0443-4afd-8255-fc72c02ee34d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:28:44.410737 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05d5d5-0443-4afd-8255-fc72c02ee34d service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:28:44.420351 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1448/2865] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:28:47.498754 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bfeaf-d48a-4b91-a73b-c60c42122e22 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:47.499303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bfeaf-d48a-4b91-a73b-c60c42122e22 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:47.499303 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27bfeaf-d48a-4b91-a73b-c60c42122e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:47.519077 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-e27bfeaf-d48a-4b91-a73b-c60c42122e22 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:28:47.520288 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-e27bfeaf-d48a-4b91-a73b-c60c42122e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:28:47.529340 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1418/2866] 10.222.0.39 () {58 vars in 1016 bytes} [Tue Jul 5 10:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:28:47.554370 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05d5d5-0443-4afd-8255-fc72c02ee34d service nova] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:28:47.569689 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05d5d5-0443-4afd-8255-fc72c02ee34d service nova] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:28:47.579135 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71da8b-2243-4ef5-bce2-b19b14a6d7cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:28:47.579788 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71da8b-2243-4ef5-bce2-b19b14a6d7cf service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:28:47.580076 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71da8b-2243-4ef5-bce2-b19b14a6d7cf service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:28:47.580351 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71da8b-2243-4ef5-bce2-b19b14a6d7cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:28:47.589308 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71da8b-2243-4ef5-bce2-b19b14a6d7cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:28:47.590958 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71da8b-2243-4ef5-bce2-b19b14a6d7cf service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:28:47.601288 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1449/2867] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:28:57.513816 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=24 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:57.596759 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=43 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:29:02.752044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86486e7e-e839-4c04-85e3-7df751148c64 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:29:02.752617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86486e7e-e839-4c04-85e3-7df751148c64 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:29:02.752617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86486e7e-e839-4c04-85e3-7df751148c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:29:02.771511 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-86486e7e-e839-4c04-85e3-7df751148c64 None None] MFA Rules not processed for user `7715154f39dc457695d5991dac30e107`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:29:02.772761 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-86486e7e-e839-4c04-85e3-7df751148c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:29:02.780952 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1419/2868] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:29:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:29:12.765881 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:29:15.565360 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544895e3-1b36-4ee0-a929-9439df7105e1 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:29:15.565975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544895e3-1b36-4ee0-a929-9439df7105e1 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:29:15.565975 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544895e3-1b36-4ee0-a929-9439df7105e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:29:15.586744 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-544895e3-1b36-4ee0-a929-9439df7105e1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:29:15.588847 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-544895e3-1b36-4ee0-a929-9439df7105e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:29:15.598716 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1450/2869] 10.222.0.39 () {58 vars in 1016 bytes} [Tue Jul 5 10:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:29:15.623061 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86486e7e-e839-4c04-85e3-7df751148c64 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:29:15.641014 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86486e7e-e839-4c04-85e3-7df751148c64 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:29:15.646856 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f08fe-24db-4322-8d41-b9533e55eb37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:29:15.647558 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f08fe-24db-4322-8d41-b9533e55eb37 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:29:15.647851 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f08fe-24db-4322-8d41-b9533e55eb37 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:29:15.648143 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f08fe-24db-4322-8d41-b9533e55eb37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:29:15.657051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f08fe-24db-4322-8d41-b9533e55eb37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:29:15.658700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f08fe-24db-4322-8d41-b9533e55eb37 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:29:15.668511 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1420/2870] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:29:18.897282 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6246230-135a-4483-abb8-b27426349a41 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:29:18.897942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6246230-135a-4483-abb8-b27426349a41 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:29:18.897942 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6246230-135a-4483-abb8-b27426349a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:29:18.916846 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-d6246230-135a-4483-abb8-b27426349a41 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:29:18.918062 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-d6246230-135a-4483-abb8-b27426349a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:29:18.927264 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1451/2871] 10.222.0.39 () {58 vars in 1016 bytes} [Tue Jul 5 10:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:29:18.950454 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f08fe-24db-4322-8d41-b9533e55eb37 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:29:18.961820 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f08fe-24db-4322-8d41-b9533e55eb37 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:29:18.967686 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb3ea0-140b-47e2-bac8-d7ed05a4644a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:29:18.968341 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb3ea0-140b-47e2-bac8-d7ed05a4644a service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:29:18.968632 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb3ea0-140b-47e2-bac8-d7ed05a4644a service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:29:18.968907 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb3ea0-140b-47e2-bac8-d7ed05a4644a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:29:18.977846 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb3ea0-140b-47e2-bac8-d7ed05a4644a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:29:18.979494 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb3ea0-140b-47e2-bac8-d7ed05a4644a service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:29:18.989514 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1421/2872] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:29:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:29:21.993421 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652096d-bf2c-4775-95f0-c6e956a3da16 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:29:21.993930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652096d-bf2c-4775-95f0-c6e956a3da16 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:29:21.993930 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652096d-bf2c-4775-95f0-c6e956a3da16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:29:22.013192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-7652096d-bf2c-4775-95f0-c6e956a3da16 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:29:22.014459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7652096d-bf2c-4775-95f0-c6e956a3da16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:29:22.026751 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1452/2873] 10.222.0.39 () {58 vars in 1016 bytes} [Tue Jul 5 10:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:29:22.048586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb3ea0-140b-47e2-bac8-d7ed05a4644a service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:29:22.060579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb3ea0-140b-47e2-bac8-d7ed05a4644a service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:29:22.067555 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a1edd-d6c3-4f8b-8dc0-d7d6d3c5e4a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:29:22.068183 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a1edd-d6c3-4f8b-8dc0-d7d6d3c5e4a2 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:29:22.068471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a1edd-d6c3-4f8b-8dc0-d7d6d3c5e4a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:29:22.068751 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a1edd-d6c3-4f8b-8dc0-d7d6d3c5e4a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:29:22.077862 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a1edd-d6c3-4f8b-8dc0-d7d6d3c5e4a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:29:22.079483 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a1edd-d6c3-4f8b-8dc0-d7d6d3c5e4a2 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:29:22.089158 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1422/2874] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:29:26.248501 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2b13d-7973-446c-9dbd-24d60536bb0b None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:29:26.249371 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2b13d-7973-446c-9dbd-24d60536bb0b None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:29:26.249579 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2b13d-7973-446c-9dbd-24d60536bb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:29:26.276252 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-5ee2b13d-7973-446c-9dbd-24d60536bb0b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:29:26.277492 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5ee2b13d-7973-446c-9dbd-24d60536bb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:29:26.287252 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1453/2875] 10.222.0.39 () {58 vars in 1016 bytes} [Tue Jul 5 10:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:29:26.310898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a1edd-d6c3-4f8b-8dc0-d7d6d3c5e4a2 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:29:26.322838 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a1edd-d6c3-4f8b-8dc0-d7d6d3c5e4a2 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:29:26.330117 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762ea26-5c8b-4cf6-9b4a-e920afa99d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:29:26.330797 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762ea26-5c8b-4cf6-9b4a-e920afa99d5a service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:29:26.331084 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762ea26-5c8b-4cf6-9b4a-e920afa99d5a service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:29:26.331362 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762ea26-5c8b-4cf6-9b4a-e920afa99d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:29:26.340118 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762ea26-5c8b-4cf6-9b4a-e920afa99d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:29:26.341700 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762ea26-5c8b-4cf6-9b4a-e920afa99d5a service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:29:26.351350 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1423/2876] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:29:30.443857 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820c900-1420-4fff-be74-716304047cb1 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:29:30.444531 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820c900-1420-4fff-be74-716304047cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:29:30.444709 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820c900-1420-4fff-be74-716304047cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:29:30.474738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-7820c900-1420-4fff-be74-716304047cb1 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:29:30.475994 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-7820c900-1420-4fff-be74-716304047cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:29:30.485560 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1454/2877] 10.222.0.39 () {58 vars in 1016 bytes} [Tue Jul 5 10:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:29:30.508995 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762ea26-5c8b-4cf6-9b4a-e920afa99d5a service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:29:30.520380 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762ea26-5c8b-4cf6-9b4a-e920afa99d5a service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:29:30.527706 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448947c-ed46-47ac-bc06-0f39291ec5b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:29:30.528418 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448947c-ed46-47ac-bc06-0f39291ec5b9 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:29:30.528923 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448947c-ed46-47ac-bc06-0f39291ec5b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:29:30.529414 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448947c-ed46-47ac-bc06-0f39291ec5b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:29:30.543828 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448947c-ed46-47ac-bc06-0f39291ec5b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:29:30.546315 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448947c-ed46-47ac-bc06-0f39291ec5b9 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:29:30.559393 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1424/2878] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:29:40.467189 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=15 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:29:40.553062 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=30 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:00.890197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820c900-1420-4fff-be74-716304047cb1 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:00.900309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820c900-1420-4fff-be74-716304047cb1 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:00.905175 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e85f34-9bcd-4176-bdcd-f597775ff055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:00.906171 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85f34-9bcd-4176-bdcd-f597775ff055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:00.906723 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85f34-9bcd-4176-bdcd-f597775ff055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:00.907087 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e85f34-9bcd-4176-bdcd-f597775ff055 admin admin] PATH_INFO: `/v3/users/7715154f39dc457695d5991dac30e107` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:00.908382 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e85f34-9bcd-4176-bdcd-f597775ff055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7715154f39dc457695d5991dac30e107)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:00.910460 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e85f34-9bcd-4176-bdcd-f597775ff055 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:00.957970 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6e85f34-9bcd-4176-bdcd-f597775ff055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7715154f39dc457695d5991dac30e107'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:00.958660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6e85f34-9bcd-4176-bdcd-f597775ff055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7715154f39dc457695d5991dac30e107'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:00.970092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6e85f34-9bcd-4176-bdcd-f597775ff055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7715154f39dc457695d5991dac30e107'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:00.977105 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6e85f34-9bcd-4176-bdcd-f597775ff055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7715154f39dc457695d5991dac30e107'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:00.977734 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-a6e85f34-9bcd-4176-bdcd-f597775ff055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7715154f39dc457695d5991dac30e107'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:00.992198 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1455/2879] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:30:00 2022] DELETE /identity/v3/users/7715154f39dc457695d5991dac30e107 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:30:01.173771 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448947c-ed46-47ac-bc06-0f39291ec5b9 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:01.188245 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448947c-ed46-47ac-bc06-0f39291ec5b9 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:01.249051 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:01.249915 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:01.250214 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:01.250500 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] PATH_INFO: `/v3/projects/dccedf0b40e94111bf3b31f9347dfa44` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:01.251677 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=dccedf0b40e94111bf3b31f9347dfa44)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:01.253605 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:01.300716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dccedf0b40e94111bf3b31f9347dfa44', 'request_id': 'req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:01.309410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dccedf0b40e94111bf3b31f9347dfa44', 'request_id': 'req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:01.315283 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dccedf0b40e94111bf3b31f9347dfa44', 'request_id': 'req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:01.327325 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] The token cache is being invalidate because project dccedf0b40e94111bf3b31f9347dfa44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:30:01.327649 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:01.328404 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-061d08d0-e0ff-4050-9e99-bd1b1c2ad7aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:01.330043 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1425/2880] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:30:01 2022] DELETE /identity/v3/projects/dccedf0b40e94111bf3b31f9347dfa44 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:30:01.335470 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864d403-bc10-4f21-81a9-88a77ced62e3 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:01.335763 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864d403-bc10-4f21-81a9-88a77ced62e3 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:01.336036 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864d403-bc10-4f21-81a9-88a77ced62e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:01.359983 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-3864d403-bc10-4f21-81a9-88a77ced62e3 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:01.361707 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3864d403-bc10-4f21-81a9-88a77ced62e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:01.437986 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1456/2881] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:30:01.446167 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2bfeb-01e9-4718-a902-60e536f32ff6 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:01.446499 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2bfeb-01e9-4718-a902-60e536f32ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:01.446804 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2bfeb-01e9-4718-a902-60e536f32ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:01.473763 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-bda2bfeb-01e9-4718-a902-60e536f32ff6 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:01.475033 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-bda2bfeb-01e9-4718-a902-60e536f32ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:01.484418 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1426/2882] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:30:01.488920 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864d403-bc10-4f21-81a9-88a77ced62e3 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:01.499396 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864d403-bc10-4f21-81a9-88a77ced62e3 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:01.505837 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237385a3-0d18-4ccb-9b84-35d4d23522fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:01.506569 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237385a3-0d18-4ccb-9b84-35d4d23522fe admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:01.506849 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237385a3-0d18-4ccb-9b84-35d4d23522fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:01.507125 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237385a3-0d18-4ccb-9b84-35d4d23522fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:01.507856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237385a3-0d18-4ccb-9b84-35d4d23522fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:30:01.508169 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237385a3-0d18-4ccb-9b84-35d4d23522fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:01.509827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237385a3-0d18-4ccb-9b84-35d4d23522fe admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:01.522073 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1457/2883] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:30:01.526526 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2bfeb-01e9-4718-a902-60e536f32ff6 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:01.535991 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2bfeb-01e9-4718-a902-60e536f32ff6 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:01.541204 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d291df-028e-47c4-b780-a5329b25a064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:01.541872 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d291df-028e-47c4-b780-a5329b25a064 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:01.542129 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d291df-028e-47c4-b780-a5329b25a064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:01.542532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d291df-028e-47c4-b780-a5329b25a064 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:01.543226 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d291df-028e-47c4-b780-a5329b25a064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:01.545124 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d291df-028e-47c4-b780-a5329b25a064 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:01.587147 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1427/2884] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:30:01 2022] POST /identity/v3/projects => generated 399 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:30:01.592940 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237385a3-0d18-4ccb-9b84-35d4d23522fe admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:01.603291 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237385a3-0d18-4ccb-9b84-35d4d23522fe admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:01.660721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d341e1-bc34-4b70-a024-1d566abbbef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:01.661319 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d341e1-bc34-4b70-a024-1d566abbbef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:01.661604 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d341e1-bc34-4b70-a024-1d566abbbef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:01.661884 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d341e1-bc34-4b70-a024-1d566abbbef8 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:01.662612 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d341e1-bc34-4b70-a024-1d566abbbef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:01.664552 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d341e1-bc34-4b70-a024-1d566abbbef8 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:01.671485 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:01.688583 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1458/2885] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:30:01 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:30:01.692673 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d291df-028e-47c4-b780-a5329b25a064 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:01.708669 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d291df-028e-47c4-b780-a5329b25a064 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:01.714898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629daad-fed9-4cda-a784-0b3f7774d41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:01.715542 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629daad-fed9-4cda-a784-0b3f7774d41f admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:01.715826 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629daad-fed9-4cda-a784-0b3f7774d41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:01.716104 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629daad-fed9-4cda-a784-0b3f7774d41f admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:01.716841 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629daad-fed9-4cda-a784-0b3f7774d41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:01.718468 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629daad-fed9-4cda-a784-0b3f7774d41f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:01.729218 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1428/2886] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:30:01.733379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d341e1-bc34-4b70-a024-1d566abbbef8 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:01.743511 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d341e1-bc34-4b70-a024-1d566abbbef8 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:01.749061 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e29820-6812-4c2d-945e-38ee58c9402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:01.749733 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29820-6812-4c2d-945e-38ee58c9402e admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:01.750009 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29820-6812-4c2d-945e-38ee58c9402e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:01.750309 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29820-6812-4c2d-945e-38ee58c9402e admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:01.751034 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e29820-6812-4c2d-945e-38ee58c9402e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:01.752638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e29820-6812-4c2d-945e-38ee58c9402e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:01.765230 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1459/2887] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:30:01.769456 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629daad-fed9-4cda-a784-0b3f7774d41f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:01.779707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629daad-fed9-4cda-a784-0b3f7774d41f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:01.784988 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ccfc3-5259-4908-be13-9f6fbea4ac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:01.785901 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ccfc3-5259-4908-be13-9f6fbea4ac46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:01.786171 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ccfc3-5259-4908-be13-9f6fbea4ac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:01.786479 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ccfc3-5259-4908-be13-9f6fbea4ac46 admin admin] PATH_INFO: `/v3/projects/9eac9b38c8904d3e9f8d45d8eb778516/users/0f37b2e130bd4abf99072504c421bf17/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:01.801589 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ccfc3-5259-4908-be13-9f6fbea4ac46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eac9b38c8904d3e9f8d45d8eb778516, user_id=0f37b2e130bd4abf99072504c421bf17, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:01.804266 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ccfc3-5259-4908-be13-9f6fbea4ac46 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:01.820180 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1429/2888] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:30:01 2022] PUT /identity/v3/projects/9eac9b38c8904d3e9f8d45d8eb778516/users/0f37b2e130bd4abf99072504c421bf17/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:30:01.834376 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b5ba8-41b1-4bc9-b6d6-458ef1cf9196 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:01.834656 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b5ba8-41b1-4bc9-b6d6-458ef1cf9196 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:01.834992 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b5ba8-41b1-4bc9-b6d6-458ef1cf9196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:01.872212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-767b5ba8-41b1-4bc9-b6d6-458ef1cf9196 None None] MFA Rules not processed for user `0f37b2e130bd4abf99072504c421bf17`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:01.873418 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-767b5ba8-41b1-4bc9-b6d6-458ef1cf9196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:01.939600 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1460/2889] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:30:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:30:01.951443 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ccfc3-5259-4908-be13-9f6fbea4ac46 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:01.953320 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5a5ccfc3-5259-4908-be13-9f6fbea4ac46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:01.974102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:02.025165 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ccfc3-5259-4908-be13-9f6fbea4ac46 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:02.029176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00aa23-6e8b-49cb-9436-fdb53d62c253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc11037d649241608ea1378cf4692dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:02.029839 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00aa23-6e8b-49cb-9436-fdb53d62c253 service cinder] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:02.030129 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00aa23-6e8b-49cb-9436-fdb53d62c253 service cinder] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:02.030440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00aa23-6e8b-49cb-9436-fdb53d62c253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:02.039321 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00aa23-6e8b-49cb-9436-fdb53d62c253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:02.040773 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00aa23-6e8b-49cb-9436-fdb53d62c253 service cinder] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:02.049914 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1430/2890] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:30:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:30:07.100762 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b5ba8-41b1-4bc9-b6d6-458ef1cf9196 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:07.166325 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b5ba8-41b1-4bc9-b6d6-458ef1cf9196 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:07.171223 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:07.172095 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:07.172379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:07.172654 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] PATH_INFO: `/v3/users/0f37b2e130bd4abf99072504c421bf17` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:07.173856 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f37b2e130bd4abf99072504c421bf17)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:07.175826 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:07.226488 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f37b2e130bd4abf99072504c421bf17'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:07.227131 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f37b2e130bd4abf99072504c421bf17'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:07.239479 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f37b2e130bd4abf99072504c421bf17'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:07.244308 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:07.247696 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f37b2e130bd4abf99072504c421bf17'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:07.248828 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f37b2e130bd4abf99072504c421bf17'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:07.263661 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1461/2891] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:30:07 2022] DELETE /identity/v3/users/0f37b2e130bd4abf99072504c421bf17 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:30:07.268527 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84fc07-45ca-4657-99d7-1a20b2d2a508 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:07.269052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84fc07-45ca-4657-99d7-1a20b2d2a508 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:07.269052 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84fc07-45ca-4657-99d7-1a20b2d2a508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:07.287766 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-ac84fc07-45ca-4657-99d7-1a20b2d2a508 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:07.289027 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ac84fc07-45ca-4657-99d7-1a20b2d2a508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:07.351537 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1431/2892] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:30:07.364126 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:07.365638 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:07.432943 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acbb1f-b5ca-4f73-9302-ce51eea3ee66 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:07.437268 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c7ef0-c535-4a2f-936f-9566df4dd91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:07.437925 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c7ef0-c535-4a2f-936f-9566df4dd91e service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:07.438212 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c7ef0-c535-4a2f-936f-9566df4dd91e service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:07.438521 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c7ef0-c535-4a2f-936f-9566df4dd91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:07.449197 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c7ef0-c535-4a2f-936f-9566df4dd91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:07.450704 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c7ef0-c535-4a2f-936f-9566df4dd91e service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:07.460807 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1462/2893] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 10:30:07.741116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84fc07-45ca-4657-99d7-1a20b2d2a508 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:07.742813 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-ac84fc07-45ca-4657-99d7-1a20b2d2a508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:07.759065 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84fc07-45ca-4657-99d7-1a20b2d2a508 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:07.768703 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:07.769588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:07.769865 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:07.770139 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] PATH_INFO: `/v3/projects/9eac9b38c8904d3e9f8d45d8eb778516` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:07.771393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eac9b38c8904d3e9f8d45d8eb778516)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:07.773420 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:07.831553 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eac9b38c8904d3e9f8d45d8eb778516', 'request_id': 'req-5e472121-85e1-4b47-892b-d7522dc7fa20'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:07.841395 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eac9b38c8904d3e9f8d45d8eb778516', 'request_id': 'req-5e472121-85e1-4b47-892b-d7522dc7fa20'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:07.847391 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eac9b38c8904d3e9f8d45d8eb778516', 'request_id': 'req-5e472121-85e1-4b47-892b-d7522dc7fa20'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:07.858582 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] The token cache is being invalidate because project 9eac9b38c8904d3e9f8d45d8eb778516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:30:07.858814 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:07.859508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-5e472121-85e1-4b47-892b-d7522dc7fa20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:30:07.861121 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1432/2894] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:30:07 2022] DELETE /identity/v3/projects/9eac9b38c8904d3e9f8d45d8eb778516 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:30:07.866376 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4b544-2c71-477c-b33a-d0e5f4612997 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:07.866738 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4b544-2c71-477c-b33a-d0e5f4612997 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:07.867040 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4b544-2c71-477c-b33a-d0e5f4612997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:07.885013 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-9ab4b544-2c71-477c-b33a-d0e5f4612997 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:07.886284 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9ab4b544-2c71-477c-b33a-d0e5f4612997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:07.947712 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1463/2895] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:30:07.955602 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e98c6-f8df-4e79-a4ee-d35e062ec27b None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:07.955977 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e98c6-f8df-4e79-a4ee-d35e062ec27b None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:07.956369 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e98c6-f8df-4e79-a4ee-d35e062ec27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:07.974973 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-232e98c6-f8df-4e79-a4ee-d35e062ec27b None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:07.976217 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-232e98c6-f8df-4e79-a4ee-d35e062ec27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:07.984984 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1433/2896] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:30:07.989422 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4b544-2c71-477c-b33a-d0e5f4612997 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:07.999755 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4b544-2c71-477c-b33a-d0e5f4612997 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:08.006787 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f8af5-b91c-4289-8e9b-dee372328836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:08.007750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f8af5-b91c-4289-8e9b-dee372328836 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:08.008065 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f8af5-b91c-4289-8e9b-dee372328836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:08.008410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f8af5-b91c-4289-8e9b-dee372328836 admin admin] PATH_INFO: `/v3/domains` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:08.009516 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f8af5-b91c-4289-8e9b-dee372328836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 10:30:08.009883 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f8af5-b91c-4289-8e9b-dee372328836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:08.012381 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f8af5-b91c-4289-8e9b-dee372328836 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:08.027621 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1464/2897] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jul 5 10:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 10:30:08.032116 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e98c6-f8df-4e79-a4ee-d35e062ec27b None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:08.041922 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e98c6-f8df-4e79-a4ee-d35e062ec27b None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:08.048093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42c0fd-b0d0-4d3e-8a16-f812c0ab1d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:08.048754 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42c0fd-b0d0-4d3e-8a16-f812c0ab1d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:08.049044 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42c0fd-b0d0-4d3e-8a16-f812c0ab1d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:08.049312 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42c0fd-b0d0-4d3e-8a16-f812c0ab1d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:08.050018 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42c0fd-b0d0-4d3e-8a16-f812c0ab1d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:08.051996 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42c0fd-b0d0-4d3e-8a16-f812c0ab1d82 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:08.080961 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1434/2898] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:30:08 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:30:08.085075 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f8af5-b91c-4289-8e9b-dee372328836 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:08.095525 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f8af5-b91c-4289-8e9b-dee372328836 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:08.154931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae806e84-6866-4f9b-a3d4-ad6a32f4bb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:08.155543 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae806e84-6866-4f9b-a3d4-ad6a32f4bb9a admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:08.155795 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae806e84-6866-4f9b-a3d4-ad6a32f4bb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:08.156066 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae806e84-6866-4f9b-a3d4-ad6a32f4bb9a admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:08.156750 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae806e84-6866-4f9b-a3d4-ad6a32f4bb9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:08.158704 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae806e84-6866-4f9b-a3d4-ad6a32f4bb9a admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:08.181883 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1465/2899] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:30:08 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:30:08.186074 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42c0fd-b0d0-4d3e-8a16-f812c0ab1d82 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:08.197516 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42c0fd-b0d0-4d3e-8a16-f812c0ab1d82 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:08.203111 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dbd7a-0089-409c-8c27-85b8a97f8271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:08.203780 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dbd7a-0089-409c-8c27-85b8a97f8271 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:08.204068 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dbd7a-0089-409c-8c27-85b8a97f8271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:08.204350 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dbd7a-0089-409c-8c27-85b8a97f8271 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:08.205088 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dbd7a-0089-409c-8c27-85b8a97f8271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:08.206755 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dbd7a-0089-409c-8c27-85b8a97f8271 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:08.217486 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1435/2900] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:30:08.221448 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae806e84-6866-4f9b-a3d4-ad6a32f4bb9a admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:08.231702 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae806e84-6866-4f9b-a3d4-ad6a32f4bb9a admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:08.236785 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761317aa-3562-436d-8e95-8dee30109824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:08.237437 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761317aa-3562-436d-8e95-8dee30109824 admin admin] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:08.237740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761317aa-3562-436d-8e95-8dee30109824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:08.238019 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761317aa-3562-436d-8e95-8dee30109824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:08.238776 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761317aa-3562-436d-8e95-8dee30109824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:08.240366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761317aa-3562-436d-8e95-8dee30109824 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:08.241754 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:08.252147 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1466/2901] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:30:08.256067 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dbd7a-0089-409c-8c27-85b8a97f8271 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:08.262504 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:08.266968 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dbd7a-0089-409c-8c27-85b8a97f8271 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:08.273389 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8637cb-8bf8-4759-adde-42f4877cca7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:08.274440 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8637cb-8bf8-4759-adde-42f4877cca7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:08.274974 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8637cb-8bf8-4759-adde-42f4877cca7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:08.275470 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8637cb-8bf8-4759-adde-42f4877cca7f admin admin] PATH_INFO: `/v3/projects/2613d243336f4fbb97c64ac93139135f/users/1ce2fa3f26e247c5a96a660ea5a0037a/roles/b10cd0627cc84a9c92dba333901e60e1` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:08.290640 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8637cb-8bf8-4759-adde-42f4877cca7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2613d243336f4fbb97c64ac93139135f, user_id=1ce2fa3f26e247c5a96a660ea5a0037a, role_id=b10cd0627cc84a9c92dba333901e60e1)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:08.293297 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8637cb-8bf8-4759-adde-42f4877cca7f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:08.308147 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1436/2902] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:30:08 2022] PUT /identity/v3/projects/2613d243336f4fbb97c64ac93139135f/users/1ce2fa3f26e247c5a96a660ea5a0037a/roles/b10cd0627cc84a9c92dba333901e60e1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:30:08.313410 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3245d4-98e9-446a-8b75-d1f49dc818a8 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:08.313730 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3245d4-98e9-446a-8b75-d1f49dc818a8 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:08.314012 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3245d4-98e9-446a-8b75-d1f49dc818a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:08.332626 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-dc3245d4-98e9-446a-8b75-d1f49dc818a8 None None] MFA Rules not processed for user `c042e1f397444e8991d3f0b7bfb28908`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:08.333827 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-dc3245d4-98e9-446a-8b75-d1f49dc818a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:08.400146 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1467/2903] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 10:30:08.413802 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8637cb-8bf8-4759-adde-42f4877cca7f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:08.415784 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-5c8637cb-8bf8-4759-adde-42f4877cca7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:08.475900 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8637cb-8bf8-4759-adde-42f4877cca7f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:08.482008 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ae37a-d66a-4fe5-854d-c91c0ccc158f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:08.483023 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ae37a-d66a-4fe5-854d-c91c0ccc158f service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:08.483291 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ae37a-d66a-4fe5-854d-c91c0ccc158f service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:08.483679 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ae37a-d66a-4fe5-854d-c91c0ccc158f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:08.498792 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ae37a-d66a-4fe5-854d-c91c0ccc158f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:08.500978 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ae37a-d66a-4fe5-854d-c91c0ccc158f service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:08.513062 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1437/2904] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:30:12.597941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfd64-6a24-4ed8-b40a-7adb4546d0f2 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:12.597941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfd64-6a24-4ed8-b40a-7adb4546d0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:12.597941 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfd64-6a24-4ed8-b40a-7adb4546d0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:12.634718 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-aebcfd64-6a24-4ed8-b40a-7adb4546d0f2 None None] MFA Rules not processed for user `1ce2fa3f26e247c5a96a660ea5a0037a`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:12.635966 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-aebcfd64-6a24-4ed8-b40a-7adb4546d0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:12.705306 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1468/2905] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:30:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:30:12.709854 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ae37a-d66a-4fe5-854d-c91c0ccc158f service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:12.720429 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ae37a-d66a-4fe5-854d-c91c0ccc158f service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:12.725698 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef40cfe-7cdd-46c9-b4bc-e35530248c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:12.726657 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef40cfe-7cdd-46c9-b4bc-e35530248c5f admin admin] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:12.726888 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef40cfe-7cdd-46c9-b4bc-e35530248c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:12.727214 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef40cfe-7cdd-46c9-b4bc-e35530248c5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:12.728007 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef40cfe-7cdd-46c9-b4bc-e35530248c5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:12.730024 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef40cfe-7cdd-46c9-b4bc-e35530248c5f admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:12.761258 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1438/2906] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jul 5 10:30:12 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 10:30:12.765180 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcfd64-6a24-4ed8-b40a-7adb4546d0f2 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:12.776362 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcfd64-6a24-4ed8-b40a-7adb4546d0f2 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:12.833853 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d360a-4b03-4b4f-9e32-aec8d6f79078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:12.834740 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d360a-4b03-4b4f-9e32-aec8d6f79078 admin admin] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:12.835167 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d360a-4b03-4b4f-9e32-aec8d6f79078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:12.835528 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d360a-4b03-4b4f-9e32-aec8d6f79078 admin admin] PATH_INFO: `/v3/users` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:12.836293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d360a-4b03-4b4f-9e32-aec8d6f79078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:12.838330 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d360a-4b03-4b4f-9e32-aec8d6f79078 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:12.862936 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1469/2907] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jul 5 10:30:12 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 10:30:12.864025 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef40cfe-7cdd-46c9-b4bc-e35530248c5f admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:12.874586 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef40cfe-7cdd-46c9-b4bc-e35530248c5f admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:12.879410 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eaab7-b9f0-484c-a626-21189ca0de3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:12.880102 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eaab7-b9f0-484c-a626-21189ca0de3b admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:12.880393 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eaab7-b9f0-484c-a626-21189ca0de3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:12.880831 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eaab7-b9f0-484c-a626-21189ca0de3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:12.881579 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8eaab7-b9f0-484c-a626-21189ca0de3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:12.883290 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8eaab7-b9f0-484c-a626-21189ca0de3b admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:12.894884 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1439/2908] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 10:30:12.899067 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d360a-4b03-4b4f-9e32-aec8d6f79078 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:12.910043 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d360a-4b03-4b4f-9e32-aec8d6f79078 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:12.914946 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341004e5-3468-4a35-801a-52491303e41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:12.915867 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341004e5-3468-4a35-801a-52491303e41e admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:12.916155 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341004e5-3468-4a35-801a-52491303e41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:12.916440 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341004e5-3468-4a35-801a-52491303e41e admin admin] PATH_INFO: `/v3/projects/2db2d4c92835467e886d74e82d97502e/users/f961d868996c4aadb5eec5a7d4e9bcea/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:12.931592 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:12.935146 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341004e5-3468-4a35-801a-52491303e41e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db2d4c92835467e886d74e82d97502e, user_id=f961d868996c4aadb5eec5a7d4e9bcea, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:12.938285 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341004e5-3468-4a35-801a-52491303e41e admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:12.953872 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1470/2909] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:30:12 2022] PUT /identity/v3/projects/2db2d4c92835467e886d74e82d97502e/users/f961d868996c4aadb5eec5a7d4e9bcea/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 10:30:12.958095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eaab7-b9f0-484c-a626-21189ca0de3b admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:12.968953 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eaab7-b9f0-484c-a626-21189ca0de3b admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:13.027058 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c4d95-4195-4a2d-b7f0-089c35d23683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:13.027770 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4d95-4195-4a2d-b7f0-089c35d23683 admin admin] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:13.028195 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4d95-4195-4a2d-b7f0-089c35d23683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:13.028321 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4d95-4195-4a2d-b7f0-089c35d23683 admin admin] PATH_INFO: `/v3/roles` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:13.029073 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c4d95-4195-4a2d-b7f0-089c35d23683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:13.030969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c4d95-4195-4a2d-b7f0-089c35d23683 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:13.042408 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1440/2910] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jul 5 10:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 10:30:13.046722 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341004e5-3468-4a35-801a-52491303e41e admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:13.058097 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341004e5-3468-4a35-801a-52491303e41e admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:13.063472 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbe1c9-7c23-452f-98e8-9bcc39b19c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:13.064427 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbe1c9-7c23-452f-98e8-9bcc39b19c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:13.064708 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbe1c9-7c23-452f-98e8-9bcc39b19c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:13.064986 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbe1c9-7c23-452f-98e8-9bcc39b19c8c admin admin] PATH_INFO: `/v3/projects/2db2d4c92835467e886d74e82d97502e/users/f961d868996c4aadb5eec5a7d4e9bcea/roles/c184a177010144429f495f3290d28221` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:13.067027 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbe1c9-7c23-452f-98e8-9bcc39b19c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db2d4c92835467e886d74e82d97502e, user_id=f961d868996c4aadb5eec5a7d4e9bcea, role_id=c184a177010144429f495f3290d28221)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:13.070069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbe1c9-7c23-452f-98e8-9bcc39b19c8c admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:13.090481 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1471/2911] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jul 5 10:30:13 2022] PUT /identity/v3/projects/2db2d4c92835467e886d74e82d97502e/users/f961d868996c4aadb5eec5a7d4e9bcea/roles/c184a177010144429f495f3290d28221 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:30:17.921668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f9923-c0c6-4b0f-a57b-e49b5362ed8c None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:17.922609 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f9923-c0c6-4b0f-a57b-e49b5362ed8c None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:17.922899 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f9923-c0c6-4b0f-a57b-e49b5362ed8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:17.970716 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-075f9923-c0c6-4b0f-a57b-e49b5362ed8c None None] MFA Rules not processed for user `f961d868996c4aadb5eec5a7d4e9bcea`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:17.973704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-075f9923-c0c6-4b0f-a57b-e49b5362ed8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:18.027532 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:18.092214 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1441/2912] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:30:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:30:18.106142 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbe1c9-7c23-452f-98e8-9bcc39b19c8c admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:18.108137 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-91dbe1c9-7c23-452f-98e8-9bcc39b19c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:18.211034 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbe1c9-7c23-452f-98e8-9bcc39b19c8c admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:18.215624 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71bdb6-2ba8-4d1f-87ce-12f84ffa07bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b97736e49e2a4bfb9cf7093cfa8b2b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:18.217854 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71bdb6-2ba8-4d1f-87ce-12f84ffa07bd service glance] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:18.218620 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71bdb6-2ba8-4d1f-87ce-12f84ffa07bd service glance] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:18.223048 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71bdb6-2ba8-4d1f-87ce-12f84ffa07bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:18.303871 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71bdb6-2ba8-4d1f-87ce-12f84ffa07bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:18.305347 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71bdb6-2ba8-4d1f-87ce-12f84ffa07bd service glance] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:18.315578 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1472/2913] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:30:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:30:18.437880 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8273c-c44e-427a-8f91-8b8edd1eb5b8 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:18.438127 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8273c-c44e-427a-8f91-8b8edd1eb5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:18.438445 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8273c-c44e-427a-8f91-8b8edd1eb5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:18.509970 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-3ef8273c-c44e-427a-8f91-8b8edd1eb5b8 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:18.511431 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-3ef8273c-c44e-427a-8f91-8b8edd1eb5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:18.578136 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1442/2914] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:30:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:30:18.625660 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71bdb6-2ba8-4d1f-87ce-12f84ffa07bd service glance] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:18.627243 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-5d71bdb6-2ba8-4d1f-87ce-12f84ffa07bd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:18.718069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71bdb6-2ba8-4d1f-87ce-12f84ffa07bd service glance] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:18.723084 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003a6b8-9f10-4b90-9a3b-ed6b410e5980 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:18.723785 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003a6b8-9f10-4b90-9a3b-ed6b410e5980 service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:18.724072 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003a6b8-9f10-4b90-9a3b-ed6b410e5980 service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:18.724434 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003a6b8-9f10-4b90-9a3b-ed6b410e5980 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:18.733442 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003a6b8-9f10-4b90-9a3b-ed6b410e5980 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:18.735140 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003a6b8-9f10-4b90-9a3b-ed6b410e5980 service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:18.745981 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1473/2915] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:30:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:30:28.204474 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210803b9-5383-4183-9fec-2857a47cbb0e None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:28.204969 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210803b9-5383-4183-9fec-2857a47cbb0e None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:28.205095 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210803b9-5383-4183-9fec-2857a47cbb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:28.223831 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-210803b9-5383-4183-9fec-2857a47cbb0e None None] MFA Rules not processed for user `1ce2fa3f26e247c5a96a660ea5a0037a`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:28.225501 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-210803b9-5383-4183-9fec-2857a47cbb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:28.233659 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1443/2916] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jul 5 10:30:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:30:28.497298 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003a6b8-9f10-4b90-9a3b-ed6b410e5980 service swift] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:28.502045 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:28.556916 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003a6b8-9f10-4b90-9a3b-ed6b410e5980 service swift] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:28.560955 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdf138-9306-46d1-9121-4334c200bceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:28.561633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdf138-9306-46d1-9121-4334c200bceb service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:28.561920 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdf138-9306-46d1-9121-4334c200bceb service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:28.562196 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdf138-9306-46d1-9121-4334c200bceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:28.564178 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4dcdf138-9306-46d1-9121-4334c200bceb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:28.574428 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdf138-9306-46d1-9121-4334c200bceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:28.575885 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdf138-9306-46d1-9121-4334c200bceb service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:28.584497 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1474/2917] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:30:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:30:34.317597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303a7bd-ae48-47c1-8c74-3df51a386d19 None None] REQUEST_METHOD: `POST` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:34.317597 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303a7bd-ae48-47c1-8c74-3df51a386d19 None None] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:34.318249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303a7bd-ae48-47c1-8c74-3df51a386d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:34.343197 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.auth.core [None req-2303a7bd-ae48-47c1-8c74-3df51a386d19 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:30:34.344983 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2303a7bd-ae48-47c1-8c74-3df51a386d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:34.353305 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1444/2918] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jul 5 10:30:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:30:34.370277 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdf138-9306-46d1-9121-4334c200bceb service neutron] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:34.389931 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdf138-9306-46d1-9121-4334c200bceb service neutron] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:34.396331 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5196fd-391b-4d82-9da0-de369059fe74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:34.397012 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5196fd-391b-4d82-9da0-de369059fe74 service swift] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:34.397301 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5196fd-391b-4d82-9da0-de369059fe74 service swift] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:34.397584 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5196fd-391b-4d82-9da0-de369059fe74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:34.407611 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5196fd-391b-4d82-9da0-de369059fe74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:34.409264 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5196fd-391b-4d82-9da0-de369059fe74 service swift] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:34.419832 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1475/2919] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:30:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:30:44.334496 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=63 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:44.417182 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=31 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:46.619931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303a7bd-ae48-47c1-8c74-3df51a386d19 None None] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:46.621537 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2303a7bd-ae48-47c1-8c74-3df51a386d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:46.705919 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303a7bd-ae48-47c1-8c74-3df51a386d19 None None] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:46.711668 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037cdb7-bacb-45db-89a5-c4b5c8a979a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:46.712313 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037cdb7-bacb-45db-89a5-c4b5c8a979a6 service nova] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:46.712600 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037cdb7-bacb-45db-89a5-c4b5c8a979a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:46.712879 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037cdb7-bacb-45db-89a5-c4b5c8a979a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:46.714931 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-7037cdb7-bacb-45db-89a5-c4b5c8a979a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:46.728358 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037cdb7-bacb-45db-89a5-c4b5c8a979a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:46.730363 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037cdb7-bacb-45db-89a5-c4b5c8a979a6 service nova] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:46.745572 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1445/2920] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:30:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:30:54.712186 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5196fd-391b-4d82-9da0-de369059fe74 service swift] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:30:54.726188 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5196fd-391b-4d82-9da0-de369059fe74 service swift] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:30:54.732944 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6663f-6a81-4de3-a37e-26ee3f3f397c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:30:54.733549 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6663f-6a81-4de3-a37e-26ee3f3f397c service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:30:54.733821 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6663f-6a81-4de3-a37e-26ee3f3f397c service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:30:54.734098 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6663f-6a81-4de3-a37e-26ee3f3f397c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:30:54.736261 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-b0e6663f-6a81-4de3-a37e-26ee3f3f397c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:30:54.749366 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6663f-6a81-4de3-a37e-26ee3f3f397c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:30:54.751568 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6663f-6a81-4de3-a37e-26ee3f3f397c service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:30:54.766007 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1476/2921] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:30:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:30:56.740559 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=34 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:31:02.113704 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037cdb7-bacb-45db-89a5-c4b5c8a979a6 service nova] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:31:02.123467 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037cdb7-bacb-45db-89a5-c4b5c8a979a6 service nova] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:31:02.127508 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b055e6-5aed-46be-92d7-802027f32b33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:31:02.128147 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b055e6-5aed-46be-92d7-802027f32b33 service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:31:02.128432 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b055e6-5aed-46be-92d7-802027f32b33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:31:02.128707 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b055e6-5aed-46be-92d7-802027f32b33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:31:02.138409 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b055e6-5aed-46be-92d7-802027f32b33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:31:02.139879 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b055e6-5aed-46be-92d7-802027f32b33 service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:31:02.153942 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1446/2922] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:31:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:31:04.760721 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=6 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:31:05.392674 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6663f-6a81-4de3-a37e-26ee3f3f397c service nova] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:31:05.409138 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6663f-6a81-4de3-a37e-26ee3f3f397c service nova] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:31:05.417906 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92976b-3d49-461c-ad83-6ed17cad0a75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee6ee8f99c34aa281bf02ecd3f23216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:31:05.418625 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92976b-3d49-461c-ad83-6ed17cad0a75 service nova] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:31:05.419148 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92976b-3d49-461c-ad83-6ed17cad0a75 service nova] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:31:05.419800 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92976b-3d49-461c-ad83-6ed17cad0a75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:31:05.423327 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-8b92976b-3d49-461c-ad83-6ed17cad0a75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:31:05.438810 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92976b-3d49-461c-ad83-6ed17cad0a75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:31:05.441356 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92976b-3d49-461c-ad83-6ed17cad0a75 service nova] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:31:05.466668 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1477/2923] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:31:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:31:12.146617 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=6 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:31:15.449037 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=6 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:31:37.727493 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b055e6-5aed-46be-92d7-802027f32b33 service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:31:37.729211 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-d6b055e6-5aed-46be-92d7-802027f32b33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:31:37.799608 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b055e6-5aed-46be-92d7-802027f32b33 service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:31:37.804808 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcac8a-7ff9-40f1-9b21-22f54f8b123f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:31:37.805453 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcac8a-7ff9-40f1-9b21-22f54f8b123f service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:31:37.805737 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcac8a-7ff9-40f1-9b21-22f54f8b123f service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:31:37.806009 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcac8a-7ff9-40f1-9b21-22f54f8b123f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:31:37.808048 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-86dcac8a-7ff9-40f1-9b21-22f54f8b123f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:31:37.819782 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcac8a-7ff9-40f1-9b21-22f54f8b123f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:31:37.821351 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcac8a-7ff9-40f1-9b21-22f54f8b123f service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:31:37.831187 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1447/2924] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:31:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:31:47.827145 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=32 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:33:24.460192 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92976b-3d49-461c-ad83-6ed17cad0a75 service nova] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:33:24.470459 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92976b-3d49-461c-ad83-6ed17cad0a75 service nova] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:33:24.474742 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e202a6e-1404-4870-abab-46b236229512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:33:24.475384 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202a6e-1404-4870-abab-46b236229512 service neutron] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:33:24.475664 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202a6e-1404-4870-abab-46b236229512 service neutron] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:33:24.475935 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202a6e-1404-4870-abab-46b236229512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:33:24.477938 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-3e202a6e-1404-4870-abab-46b236229512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:33:24.491613 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e202a6e-1404-4870-abab-46b236229512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:33:24.493110 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e202a6e-1404-4870-abab-46b236229512 service neutron] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:33:24.504444 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1478/2925] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:33:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:33:29.972087 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcac8a-7ff9-40f1-9b21-22f54f8b123f service placement] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:33:29.987246 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcac8a-7ff9-40f1-9b21-22f54f8b123f service placement] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:33:29.991941 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df376402-81fb-457f-8ac2-0084daf98a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0865f21da0a4c95a84c0099a1b9a78b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:33:29.992588 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df376402-81fb-457f-8ac2-0084daf98a1f service neutron] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:33:29.992868 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df376402-81fb-457f-8ac2-0084daf98a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:33:29.993142 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df376402-81fb-457f-8ac2-0084daf98a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:33:29.995146 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-df376402-81fb-457f-8ac2-0084daf98a1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:33:30.008641 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df376402-81fb-457f-8ac2-0084daf98a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:33:30.010099 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df376402-81fb-457f-8ac2-0084daf98a1f service neutron] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:33:30.021699 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1448/2926] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:33:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:33:32.416578 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729a358-8ef7-46e2-80ba-37e703ea9e98 None None] REQUEST_METHOD: `POST` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:33:32.417077 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729a358-8ef7-46e2-80ba-37e703ea9e98 None None] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:33:32.417200 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729a358-8ef7-46e2-80ba-37e703ea9e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:33:32.435778 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.auth.core [None req-4729a358-8ef7-46e2-80ba-37e703ea9e98 None None] MFA Rules not processed for user `84a8d62afe964cb49acc236850adbbf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 10:33:32.436971 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4729a358-8ef7-46e2-80ba-37e703ea9e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:33:32.444787 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1479/2927] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jul 5 10:33:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 10:33:32.486569 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df376402-81fb-457f-8ac2-0084daf98a1f service neutron] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:33:32.503728 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df376402-81fb-457f-8ac2-0084daf98a1f service neutron] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:33:32.511158 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b0a25-ec1a-417b-9294-fc52c3a48746 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfded29b70140a387f7b8c07c296861', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:33:32.511809 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0a25-ec1a-417b-9294-fc52c3a48746 service swift] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:33:32.512090 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0a25-ec1a-417b-9294-fc52c3a48746 service swift] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:33:32.512366 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0a25-ec1a-417b-9294-fc52c3a48746 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:33:32.521094 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b0a25-ec1a-417b-9294-fc52c3a48746 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:33:32.522739 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b0a25-ec1a-417b-9294-fc52c3a48746 service swift] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:33:32.531097 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1449/2928] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jul 5 10:33:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 10:33:36.009000 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729a358-8ef7-46e2-80ba-37e703ea9e98 None None] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:33:36.012091 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-4729a358-8ef7-46e2-80ba-37e703ea9e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:33:36.033918 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729a358-8ef7-46e2-80ba-37e703ea9e98 None None] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:33:36.051769 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e411d-fe21-4375-98a3-fedcac143025 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:33:36.052522 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e411d-fe21-4375-98a3-fedcac143025 service placement] REQUEST_METHOD: `GET` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:33:36.052735 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e411d-fe21-4375-98a3-fedcac143025 service placement] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:33:36.052998 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e411d-fe21-4375-98a3-fedcac143025 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:33:36.055042 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-9b2e411d-fe21-4375-98a3-fedcac143025 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:33:36.070374 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e411d-fe21-4375-98a3-fedcac143025 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:33:36.074092 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e411d-fe21-4375-98a3-fedcac143025 service placement] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:33:36.095083 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1480/2929] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:33:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:33:39.287093 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b0a25-ec1a-417b-9294-fc52c3a48746 service swift] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:33:39.353898 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b0a25-ec1a-417b-9294-fc52c3a48746 service swift] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:33:39.359156 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:33:39.360145 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:33:39.360347 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:33:39.360665 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] PATH_INFO: `/v3/users/1ce2fa3f26e247c5a96a660ea5a0037a` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:33:39.362316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce2fa3f26e247c5a96a660ea5a0037a)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:33:39.364176 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:33:39.415434 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce2fa3f26e247c5a96a660ea5a0037a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.416181 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce2fa3f26e247c5a96a660ea5a0037a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.416897 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce2fa3f26e247c5a96a660ea5a0037a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.429317 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce2fa3f26e247c5a96a660ea5a0037a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.447524 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce2fa3f26e247c5a96a660ea5a0037a'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.463019 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1450/2930] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:33:39 2022] DELETE /identity/v3/users/1ce2fa3f26e247c5a96a660ea5a0037a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:33:39.642963 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e411d-fe21-4375-98a3-fedcac143025 service placement] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:33:39.655842 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e411d-fe21-4375-98a3-fedcac143025 service placement] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:33:39.716104 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:33:39.716989 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:33:39.717293 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:33:39.717555 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] PATH_INFO: `/v3/projects/2613d243336f4fbb97c64ac93139135f` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:33:39.718765 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2613d243336f4fbb97c64ac93139135f)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:33:39.721074 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:33:39.768811 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2613d243336f4fbb97c64ac93139135f', 'request_id': 'req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.774083 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2613d243336f4fbb97c64ac93139135f', 'request_id': 'req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.780379 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2613d243336f4fbb97c64ac93139135f', 'request_id': 'req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.794508 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] The token cache is being invalidate because project 2613d243336f4fbb97c64ac93139135f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:33:39.794791 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.795429 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.797009 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1481/2931] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:33:39 2022] DELETE /identity/v3/projects/2613d243336f4fbb97c64ac93139135f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:33:39.801307 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:33:39.802790 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:33:39.816219 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fba83-a07c-4ec8-ba2c-cbaf1de610b8 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:33:39.863659 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:33:39.873308 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:33:39.874205 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:33:39.874530 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:33:39.874819 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] PATH_INFO: `/v3/users/f961d868996c4aadb5eec5a7d4e9bcea` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:33:39.876316 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f961d868996c4aadb5eec5a7d4e9bcea)` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:33:39.878442 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:33:39.926607 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f961d868996c4aadb5eec5a7d4e9bcea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.927256 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f961d868996c4aadb5eec5a7d4e9bcea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.927887 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f961d868996c4aadb5eec5a7d4e9bcea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.934941 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f961d868996c4aadb5eec5a7d4e9bcea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.947471 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.notifications [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f961d868996c4aadb5eec5a7d4e9bcea'} {{(pid=80377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:39.962914 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1451/2932] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jul 5 10:33:39 2022] DELETE /identity/v3/users/f961d868996c4aadb5eec5a7d4e9bcea => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:33:40.139221 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] Authenticating user token {{(pid=80376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:33:40.140744 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.fernet_utils [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:33:40.153631 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b8bff-0677-4b7a-a7a3-61f11a5f7ac7 admin admin] Validating token access rules against request {{(pid=80376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:33:40.206059 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:33:40.213465 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c042e1f397444e8991d3f0b7bfb28908', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95874cd0621e46bbbfb452bfbd474389', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:33:40.214713 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:33:40.215015 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:33:40.215359 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] PATH_INFO: `/v3/projects/2db2d4c92835467e886d74e82d97502e` {{(pid=80376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:33:40.216924 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2db2d4c92835467e886d74e82d97502e)` {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:33:40.219069 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] RBAC: Authorization granted {{(pid=80376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:33:40.266365 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2db2d4c92835467e886d74e82d97502e', 'request_id': 'req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:40.271813 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2db2d4c92835467e886d74e82d97502e', 'request_id': 'req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:40.277888 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2db2d4c92835467e886d74e82d97502e', 'request_id': 'req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538'} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:40.292633 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] The token cache is being invalidate because project 2db2d4c92835467e886d74e82d97502e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 10:33:40.292927 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:40.293684 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG keystone.notifications [None req-30fe123d-b8bb-46c1-a3ad-622ebfa2a538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 10:33:40.295393 dv-cinder-master-3-smb devstack@keystone.service[80376]: [pid: 80376|app: 0|req: 1482/2933] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jul 5 10:33:40 2022] DELETE /identity/v3/projects/2db2d4c92835467e886d74e82d97502e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 10:33:49.959457 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:33:50.289668 dv-cinder-master-3-smb devstack@keystone.service[80376]: DEBUG dbcounter [-] [80376] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:36:42.768170 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] Authenticating user token {{(pid=80377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 10:36:42.770479 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:36:42.930803 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a570f94-b617-4094-8ac0-57257d62a6d6 admin admin] Validating token access rules against request {{(pid=80377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 10:36:42.996536 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48a7b2-6ec8-4ee7-b079-321b57874a22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9d4a50f842a48b1875900426c6a1ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35af178e36e243588f642edb57180d9a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 10:36:42.997189 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48a7b2-6ec8-4ee7-b079-321b57874a22 service placement] REQUEST_METHOD: `GET` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 10:36:42.997475 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48a7b2-6ec8-4ee7-b079-321b57874a22 service placement] SCRIPT_NAME: `/identity` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 10:36:42.997745 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48a7b2-6ec8-4ee7-b079-321b57874a22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 10:36:42.999740 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.fernet_utils [None req-db48a7b2-6ec8-4ee7-b079-321b57874a22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 10:36:43.011249 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48a7b2-6ec8-4ee7-b079-321b57874a22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 10:36:43.012882 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48a7b2-6ec8-4ee7-b079-321b57874a22 service placement] RBAC: Authorization granted {{(pid=80377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 10:36:43.022883 dv-cinder-master-3-smb devstack@keystone.service[80377]: [pid: 80377|app: 0|req: 1452/2934] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jul 5 10:36:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 10:36:53.018950 dv-cinder-master-3-smb devstack@keystone.service[80377]: DEBUG dbcounter [-] [80377] Writing DB stats keystone:SELECT=89 {{(pid=80377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}